An AI intelligent monitoring and management system

Abstract

The present invention provides an AI intelligent monitoring and management system, which relates to the field of intelligent control technology, including: a data acquisition module, which is used to capture video data of a monitoring area in real time through a camera; a data processing module, which is used to decode the video data, convert the compressed video data into an image frame sequence, pre-process each frame of the image, and use a deep learning model to perform target detection on the pre-processed image frame to identify the target object in the image; a dynamic adjustment module, which is used to dynamically determine the judgment threshold according to the real-time behavior characteristics of the target object and a preset dynamic adjustment factor; a data analysis module, which is used to analyze the motion trajectory, speed and behavior pattern of the target object, and predict the behavior of the target object. The present invention can improve the accuracy, flexibility and automation of the monitoring system.

Description

Technical Field

[0001] The present invention relates to the field of intelligent control technology, and in particular to an AI intelligent monitoring and management system. Background Art

[0002] In recent years, intelligent monitoring technology based on artificial intelligence (AI) has received widespread attention and application. This type of technology can analyze surveillance videos in real time through algorithms such as deep learning, automatically identify target objects, and model and predict their behaviors. However, existing AI intelligent monitoring technology still has some problems and challenges in practical applications.

[0003] First, the complexity and diversity of the monitoring environment make target detection and behavior recognition difficult. For example, lighting conditions, occlusion, and changes in the target object's posture will affect the accuracy of recognition. Second, most existing monitoring systems use a fixed judgment threshold to determine whether the target object's behavior is normal. This method is difficult to adapt to changes in different scenarios and needs. In addition, early warning and emergency response for abnormal behavior also need to be further improved and optimized. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide an AI intelligent monitoring and management system that can improve the accuracy, flexibility and automation of the monitoring system.

[0005] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0006] An AI intelligent monitoring and management system, comprising:

[0007] A data acquisition module is used to capture video data of the monitored area in real time through a camera;

[0008] The data processing module is used to decode the video data, convert the compressed video data into an image frame sequence, preprocess each frame of the image, and use a deep learning model to perform target detection on the preprocessed image frame to identify the target object in the image;

[0009] A dynamic adjustment module, used to dynamically determine the judgment threshold according to the real-time behavior characteristics of the target object and a preset dynamic adjustment factor;

[0010] A data analysis module is used to analyze the movement trajectory, speed and behavior pattern of the target object, predict the behavior of the target object, and determine whether the behavior of the target object conforms to a normal mode or an abnormal mode based on a judgment threshold;

[0011] The early warning module is used to trigger the early warning mechanism and generate and send early warning information when the data analysis module detects that the behavior pattern of the target object matches the preset abnormal behavior pattern;

[0012] The control module is used to automatically initiate corresponding emergency response measures according to the level of the warning information.

[0013] Furthermore, the video data is decoded, the compressed video data is converted into an image frame sequence, and each frame of the image is preprocessed, including:

[0014] Reading a compressed video file or data stream, and inputting the compressed video data into a decoder for decoding operation to obtain original image block data;

[0015] According to the frame sequence information obtained during the decoding process, the original image block data is rearranged in a playback order to form a continuous image frame sequence;

[0016] For each image frame formed after decoding, perform color space conversion;

[0017] After the color space conversion is completed, create a blank image of the same size as the original image frame, starting from the upper left corner of the image, and traverse each pixel of the image frame;

[0018] For each pixel in the image, the total edge strength and edge direction are calculated according to its edge strength in the horizontal direction and the vertical direction to obtain the edge information of each pixel in the image;

[0019] The calculated edge strength value is set as the pixel value of the output image at the corresponding position, and all pixels in the image frame are traversed and processed until the entire image frame is processed, and finally the filtered image frame is output.

[0020] Furthermore, according to the real-time behavior characteristics of the target object and the preset dynamic adjustment factor, the judgment threshold is dynamically determined, including:

[0021] Identify key behavioral characteristics, including speed, acceleration, and steering frequency;

[0022] Set a base threshold for each key behavioral trait;

[0023] For each key behavior feature, a dynamic adjustment amount is calculated according to the current value of the key behavior feature and the basic threshold, as well as a preset dynamic adjustment factor;

[0024] According to the importance of each behavior feature, the corresponding weight is determined, and the corresponding weight is used to perform weighted summation on each dynamic adjustment amount to obtain a total dynamic adjustment value;

[0025] The total dynamic adjustment value is added to the basic threshold to obtain a new judgment threshold.

[0026] Furthermore, the target object's motion trajectory, speed and behavior pattern are analyzed to predict the target object's behavior, including:

[0027] Locate the target object in each frame and record the position of the target object in the image. As time goes by, the position points of the target object form a motion trajectory.

[0028] For two adjacent frames of images, the displacement of the object on the image plane is calculated according to the position change of the target object in the image;

[0029] Convert displacement to velocity according to the frame rate of the image;

[0030] Extract key features from motion trajectory and velocity data and determine the behavior patterns of target objects;

[0031] Predict the future behavior of the target object based on its current state.

[0032] Furthermore, the target object is located in each frame, and the position of the target object in the image is recorded. As time goes by, the position points of the target object form a motion trajectory, including:

[0033] In each frame, the location of the target object is determined and a series of bounding boxes are output. Each bounding box corresponds to a detected target object. The bounding box marks the location of the object in the image, the category of the object, and the confidence of the detection.

[0034] Determine a preset confidence threshold, and obtain a bounding box with a confidence greater than the confidence threshold to obtain a filtered bounding box;

[0035] Extract the center point coordinates of the filtered bounding box, where the center point coordinates represent the position of the target object in the image frame;

[0036] The coordinates of each center point are associated with the timestamp of the frame image, and as the video frames are processed continuously, a series of target object position points associated with the timestamp are obtained;

[0037] The target object position points are connected in sequence according to the order of timestamps to form the motion trajectory of the target object.

[0038] Furthermore, for two adjacent frames of images, the displacement of the object on the image plane is calculated according to the position change of the target object in the image, including:

[0039] In the first frame image, the target object is located and the coordinates of the center point of the target object's bounding box are determined. The coordinates of the center point are (x1, y1), where x1 is the horizontal coordinate and y1 is the vertical coordinate;

[0040] In the second frame image, the target object is located and the center point coordinates of the target object's bounding box are determined. The center point coordinates are (x2, y2);

[0041] The change △x of the horizontal and vertical coordinates is calculated by subtracting the horizontal coordinate x1 of the first frame from the horizontal coordinate x2 of the second frame;

[0042] The change △y of the ordinate is calculated by subtracting the ordinate y1 of the first frame from the ordinate y2 of the second frame;

[0043] According to the changes in the horizontal and vertical coordinates and the vertical coordinates, the displacement (△x, △y) of the target object on the image plane is determined.

[0044] Furthermore, key features are extracted from the motion trajectory and speed data, and the behavior pattern of the target object is determined, including:

[0045] Extract key features from motion trajectories;

[0046] Screen key features to identify those related to the target object’s behavior patterns;

[0047] Cluster similar motion trajectories and speed features into the same group and perform clustering to obtain clustering results;

[0048] Based on the clustering results, the target object's motion trajectory is divided into different behavior pattern categories;

[0049] According to the clustering results, a behavior pattern label is assigned to each category of motion trajectory.

[0050] Furthermore, the key features include the total length, average curvature, maximum curvature, and number of turning points of the trajectory.

[0051] Furthermore, judging whether the behavior of the target object conforms to a normal mode or an abnormal mode according to the judgment threshold includes:

[0052] Obtain the current judgment threshold from the dynamic adjustment module;

[0053] The target object's behavior data is compared with the judgment threshold. If all the target object's behavior data are ≤ the corresponding judgment threshold range, the target object's behavior is considered to be in a normal state. If any of the target object's behavior data is > the corresponding judgment threshold range, the target object's behavior is considered to be abnormal.

[0054] Furthermore, when the data analysis module detects that the behavior pattern of the target object matches the preset abnormal behavior pattern, the early warning mechanism is triggered to generate and send early warning information, including:

[0055] The data analysis module monitors the behavior pattern of the target object, and compares the current behavior with the patterns in the preset abnormal behavior pattern library. If it is detected to be similar to the preset abnormal behavior pattern, the similarity score is calculated to evaluate the matching degree. If the similarity is greater than the preset threshold, the current behavior of the target object is considered abnormal;

[0056] When the behavior is confirmed to be abnormal, the data analysis module will trigger the early warning mechanism, which includes:

[0057] Record all relevant information about abnormal events, including timestamps, target object status, and behavior data;

[0058] Automatically generate an alert message that includes a description of the abnormal behavior, the time and location of the occurrence;

[0059] Sent to relevant personnel via communication channels.

[0060] The above solution of the present invention includes at least the following beneficial effects:

[0061] Through the data acquisition module, the platform can monitor the designated area in real time and capture video data. The data processing module can efficiently decode and pre-process the video data, and use the deep learning model to accurately identify the target object in the image, greatly improving the efficiency and accuracy of data processing.

[0062] The dynamic adjustment module can dynamically determine the judgment threshold based on the real-time behavioral characteristics of the target object and the preset dynamic adjustment factor. This dynamic adjustment mechanism enables the system to adapt more flexibly to changes in different environments and behavioral patterns, thereby improving the adaptability and accuracy of the monitoring system.

[0063] The data analysis module can conduct in-depth analysis of the target object's movement trajectory, speed and behavior patterns, and model and predict the target object's behavior based on these analyses. This precise behavior analysis not only helps to detect abnormal behavior in a timely manner.

[0064] When the data analysis module detects that the behavior pattern of the target object matches the preset abnormal behavior pattern, the early warning module will immediately trigger the early warning mechanism, generate and send early warning information, and the control module will automatically initiate corresponding emergency response measures according to the level of the early warning information, thereby achieving a rapid and accurate response to abnormal situations, effectively improving the timeliness and effectiveness of security monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a schematic diagram of the AI ​​intelligent monitoring management system provided by an embodiment of the present invention.

[0066] Figure 2The present invention provides a schematic diagram of a process of decoding video data, converting compressed video data into an image frame sequence, and preprocessing each frame of the image.

[0067] Figure 3 It is a schematic diagram of a flow chart of dynamically determining a judgment threshold value according to the real-time behavior characteristics of a target object and a preset dynamic adjustment factor provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0069] like Figure 1 As shown, an embodiment of the present invention proposes an AI intelligent monitoring management system, including:

[0070] A data acquisition module is used to capture video data of the monitored area in real time through a camera;

[0071] The data processing module is used to decode the video data, convert the compressed video data into an image frame sequence, preprocess each frame of the image, and use a deep learning model to perform target detection on the preprocessed image frame to identify the target object in the image;

[0072] A dynamic adjustment module, used to dynamically determine the judgment threshold according to the real-time behavior characteristics of the target object and a preset dynamic adjustment factor;

[0073] A data analysis module is used to analyze the movement trajectory, speed and behavior pattern of the target object, predict the behavior of the target object, and determine whether the behavior of the target object conforms to a normal mode or an abnormal mode based on a judgment threshold;

[0074] The early warning module is used to trigger the early warning mechanism and generate and send early warning information when the data analysis module detects that the behavior pattern of the target object matches the preset abnormal behavior pattern;

[0075] The control module is used to automatically initiate corresponding emergency response measures according to the level of the warning information.

[0076] In the embodiment of the present invention, through the data acquisition module, the platform can monitor the designated area in real time and capture video data. The data processing module can efficiently decode and preprocess the video data, and accurately identify the target object in the image using the deep learning model, which greatly improves the efficiency and accuracy of data processing. The dynamic adjustment module can dynamically determine the judgment threshold according to the real-time behavior characteristics of the target object and the preset dynamic adjustment factor. This dynamic adjustment mechanism enables the system to adapt to the changes of different environments and behavior patterns more flexibly, and improves the adaptability and accuracy of the monitoring system. The data analysis module can conduct in-depth analysis of the motion trajectory, speed and behavior pattern of the target object, and model and predict the behavior of the target object based on these analyses. This accurate behavior analysis not only helps to discover abnormal behavior in time. When the data analysis module detects that the behavior pattern of the target object matches the preset abnormal behavior pattern, the early warning module will immediately trigger the early warning mechanism, generate and send early warning information, and the control module will automatically start the corresponding emergency response measures according to the level of the early warning information, thereby achieving a rapid and accurate response to abnormal situations, effectively improving the timeliness and effectiveness of security monitoring.

[0077] In another preferred embodiment of the present invention, real-time video data of the monitoring area is captured by a camera, including:

[0078] Determine the boundaries and key areas of the monitoring area, which can be represented by maps and floor plans;

[0079] Determine the visual range (viewing angle, focal length, etc.), installation location, direction, etc. of each camera;

[0080] Divide the monitoring area into a series of small grids. For each camera, calculate the number of grids it can cover. The union of these grids represents the total coverage of the camera. Identify the grids covered by multiple cameras at the same time and calculate the proportion of the overlapping area to the entire monitoring area.

[0081] Each possible camera layout is regarded as a "particle" and the optimal layout is calculated, which includes: initializing a particle swarm, where each particle represents a camera layout solution, and each particle contains parameters such as the camera position and orientation. During initialization, these parameters can be randomly selected within a certain range; for each particle (i.e., each layout solution), Calculate its fitness value F, where A and B are the coverage and repetition rate of the camera respectively. and is the weight factor;

[0082] Each particle has its own speed and position. The update formulas for speed and position are as follows:

[0083] ;

[0084] ;

[0085] in, and Particles At the moment speed and position; and Particles At the moment speed and position; is the inertia weight, and is the learning factor, and is a random number in the range [0, 1], It is a particle The individual optimal position of is the global optimal position of the entire particle swarm, i represents the index of the i-th particle in the particle swarm, and t represents the number of iterations.

[0086] In each iteration step, the individual optimal position of each particle is updated and the global optimal position of the entire particle swarm If the fitness value of the current particle is higher than its individual optimal value, its individual optimal position and the corresponding fitness value are updated. If the individual optimal fitness value of a particle is higher than the global optimal fitness value, the global optimal position and the corresponding fitness value are updated; when the preset maximum number of iterations is reached, the global optimal position is output as the optimal camera layout solution, which is the best layout that maximizes coverage while minimizing overlap.

[0087] For example, a rectangular area is 100 meters long and 50 meters wide, and contains multiple streets, restaurants, and entertainment facilities. In order to ensure comprehensive monitoring coverage and reduce blind spots while avoiding overlap between cameras, the following strategy was decided to determine the optimal layout of the cameras.

[0088] Specifically: draw a floor plan of the shopping center, including key areas such as streets, corridors, and entrances. Key areas such as cash registers, doorways, and valuable merchandise display areas are specially marked; select a camera model suitable for the shopping center environment, with a visual range of 80° wide angle and adjustable focal length; preset the camera installation height to 3 meters to ensure a wide field of view and not easily blocked; divide the shopping center area into 1×1 meter grids, a total of 5,000 grids, and for each potential camera position, calculate the number of grids it can cover to determine the initial coverage rate, identify and calculate the number of grids overlapped by multiple camera fields of view, and calculate the overlap rate based on the number of overlapping grids. Initialize a particle swarm containing 100 particles, each particle represents a camera layout plan, including the camera's position and orientation information, and set the fitness function F, where α=0.6, β=0.4, to reflect the trade-off between coverage and overlap; in each iteration, adjust the position and speed of each particle according to the speed and position update formula, and update the individual optimal position pt of each particle. i and the global optimal position g t , set the maximum number of iterations to 100; after reaching the maximum number of iterations, output the global optimal position gt as the optimal camera layout solution; according to this solution, install cameras in the shopping mall to ensure maximum coverage while minimizing overlap. The shopping mall successfully found a camera layout solution that can fully monitor key areas and reduce field of view overlap. This not only improves monitoring efficiency, but also reduces data processing and storage costs.

[0089] like Figure 2 As shown, in the embodiment of the present invention, video decoding is performed on the video data, the compressed video data is converted into an image frame sequence, and each frame of the image is preprocessed, including:

[0090] Reading a compressed video file or data stream, and inputting the compressed video data into a decoder for decoding operation to obtain original image block data;

[0091] According to the frame sequence information obtained during the decoding process, the original image block data is rearranged in a playback order to form a continuous image frame sequence;

[0092] For each image frame formed after decoding, perform color space conversion;

[0093] After the color space conversion is completed, create a blank image of the same size as the original image frame, starting from the upper left corner of the image, and traverse each pixel of the image frame;

[0094] For each pixel in the image, the total edge strength and edge direction are calculated according to its edge strength in the horizontal direction and the vertical direction to obtain the edge information of each pixel in the image;

[0095] The calculated edge strength value is set as the pixel value of the output image at the corresponding position, and all pixels in the image frame are traversed and processed until the entire image frame is processed, and finally the filtered image frame is output.

[0096] In an embodiment of the present invention, by reading a compressed video file or data stream and inputting it into a decoder for decoding, the original image block data can be efficiently restored. Subsequently, the image blocks are rearranged according to the frame order information to form a continuous and complete image frame sequence. The color space is converted for each decoded image frame, so that the image data can adapt to different processing requirements. The conversion of the color space helps to extract useful information in the image and enhance the contrast and visual effect of the image. By calculating the edge strength and edge direction of each pixel in the image, the edge information of the image can be accurately obtained. This information is crucial for tasks such as target detection, shape recognition and image segmentation. By extracting edge information, the target object in the image can be more accurately identified, and the recognition accuracy and response speed of the monitoring system can be improved. The calculated edge strength value is set as the pixel value of the output image at the corresponding position, and by traversing and processing all pixels in the image frame, the filtered image frame is finally output. This filtering process can remove noise and interference information in the image and improve the quality of the image.

[0097] In another preferred embodiment of the present invention, when used specifically, the following steps may be included:

[0098] Use a video decoding library (such as FFmpeg, GStreamer, etc.) to open and read the compressed video file or data stream; send the read compressed video data block to the decoder; the decoder performs decoding operations according to the compression format of the video, and finally outputs the decoded original image block data. According to the frame order information obtained during the decoding process, the original image block data is rearranged in the playback order to form a continuous image frame sequence; the decoder will provide frame order information during the decoding process, such as I frame (key frame), P frame (predicted frame) and B frame (bidirectional predicted frame), etc. Based on this information, the decoded image block data is rearranged in the correct playback order to ensure the continuity and correctness of the video frame. The decoded image frame may be in the YUV color space, while subsequent processing may require the RGB color space.

[0099] After the color space conversion is completed, create a blank image of the same size as the original image frame, determine the width and height of the original image frame, create a new blank image with the same size as the original image frame, and store the subsequent edge strength information; set two nested loops to traverse the rows and columns of the image respectively, and obtain its color value (R, G, B three components in RGB space) for each pixel position. For example, the Sobel operator calculates the edge strength by convolving the convolution kernel with the image. The total edge strength can be calculated by the square root of the sum of the squares of the horizontal and vertical edge strengths, and the edge direction can be determined by the ratio of the horizontal and vertical edge strengths.

[0100] The calculated edge intensity value (which may be a grayscale value) is assigned to the pixel at the corresponding position of the newly created blank image. This process converts the original image into an edge intensity image.

[0101] Continue the above traversal process until all pixels in the image frame have been processed. The final image is an image representing the edge strength of the original image, which can be used as the basic data for subsequent image analysis, feature extraction and other tasks.

[0102] like Figure 3 As shown, in an embodiment of the present invention, the judgment threshold is dynamically determined according to the real-time behavior characteristics of the target object and a preset dynamic adjustment factor, including:

[0103] Identify key behavioral characteristics, including speed, acceleration, and steering frequency;

[0104] Set a base threshold for each key behavioral trait;

[0105] For each key behavior feature, a dynamic adjustment amount is calculated according to the current value of the key behavior feature and the basic threshold, as well as a preset dynamic adjustment factor;

[0106] According to the importance of each behavior feature, the corresponding weight is determined, and the corresponding weight is used to perform weighted summation on each dynamic adjustment amount to obtain a total dynamic adjustment value;

[0107] The total dynamic adjustment value is added to the basic threshold to obtain a new judgment threshold.

[0108] In an embodiment of the present invention, by dynamically adjusting the judgment threshold, the method can make a more accurate judgment based on the real-time behavior characteristics of the target object. Compared with the traditional fixed threshold method, this method is more adaptable to the diversity and variability of the target object's behavior, thereby improving the accuracy and flexibility of the judgment. By setting a dynamic adjustment factor, the method can adjust the judgment threshold in real time according to the specific environment and situation. By determining key behavior characteristics, such as speed, acceleration, and steering frequency, and setting a basic threshold for them, the key behavior changes of the target object can be captured more accurately. Determining the corresponding weight according to the importance of each behavior feature can more reasonably reflect the influence of different behavior characteristics in judging the behavior pattern of the target object. The dynamic adjustment value obtained by weighted summation improves the accuracy of the judgment. The method of dynamically adjusting the judgment threshold can be applied to an intelligent early warning system. By adjusting the threshold in real time, the system can more accurately identify abnormal behavior and trigger the early warning mechanism in time, thereby improving the intelligence level of the early warning system.

[0109] In another preferred embodiment of the present invention, when specifically applied, it may include:

[0110] Determine key behavior characteristics, including speed v, acceleration a and steering frequency f as key behavior characteristics; set basic thresholds, including: setting a basic threshold T for speed v v ; Set the basic threshold T for acceleration a a ; Set the basic threshold T for the steering frequency f f ;

[0111] Calculate the dynamic adjustment amount, for speed, according to the current speed value v c , basic threshold T v and dynamic adjustment factor F v , calculate the dynamic adjustment △T v :

[0112] △T v =F v ×(v c -T v );

[0113] For acceleration, according to the current acceleration value a c , basic threshold T a and dynamic adjustment factor F a , calculate the dynamic adjustment amount △Ta:

[0114] △T a =F a ×(a c -T a );

[0115] For the steering frequency, according to the current steering frequency value fc , basic threshold T f and dynamic adjustment factor F f , calculate the dynamic adjustment △T f :

[0116] △T f =F f ×(f c -T f );

[0117] Determine the weight and sum it, setting the speed weight to w v , the weight of acceleration is w a , the weight of the steering frequency is w f , weighted sum to get the total dynamic adjustment value △T total :

[0118] △T total =w v ×△T v +w a ×△T a +w f ×△T f ;

[0119] Calculate the new judgment threshold, the new speed judgment threshold Tv new :in,

[0120] TV new =T v +△T total ;

[0121] Similarly, new judgment thresholds can be calculated for acceleration and steering frequency.

[0122] In an embodiment of the present invention, by dynamically adjusting the judgment threshold, the motion state of the target can be more accurately identified, including normal walking, fast movement, sudden turning, etc., thereby improving the accuracy of detection. Abnormal behavior of the target may indicate potential safety risks, such as running, sudden change of direction, etc. The method of dynamically adjusting the threshold makes the system more sensitive to the detection of these abnormal behaviors, which helps to promptly discover and respond to possible safety hazards. By dynamically adjusting the threshold, it is possible to better adapt to environmental changes, reduce false alarms and missed alarms caused by environmental changes, and improve the reliability of the detection system. By dynamically adjusting the judgment threshold of target detection, the number and motion state of the target can be more accurately estimated, thereby more effectively performing flow control and safety warnings.

[0123] In an embodiment of the present invention, the motion trajectory, speed and behavior pattern of the target object are analyzed to predict the behavior of the target object, including:

[0124] Locate the target object in each frame and record the position of the target object in the image. As time goes by, the position points of the target object form a motion trajectory.

[0125] For two adjacent frames of images, the displacement of the object on the image plane is calculated according to the position change of the target object in the image;

[0126] Convert displacement to velocity according to the frame rate of the image;

[0127] Extract key features from motion trajectory and velocity data and determine the behavior patterns of target objects;

[0128] Predict the future behavior of the target object based on its current state.

[0129] In an embodiment of the present invention, a clear and accurate motion trajectory can be formed by accurately locating the target object in each frame and recording its position; by calculating the position change of the target object in two adjacent frames of images, the displacement of the target object on the image plane can be accurately obtained, and converted into speed according to the frame rate of the image. The key features extracted from the motion trajectory and speed data can reveal the behavior pattern of the target object. Based on the current state of the target object, the system can predict the future behavior of the target object. This predictive capability enables the monitoring system to respond in advance, thereby improving the efficiency and accuracy of security monitoring. Through in-depth analysis and accurate prediction of the behavior of the target object, this method significantly improves the intelligent level of security monitoring. The system can automatically identify abnormal situations, issue warnings in a timely manner, and take corresponding emergency response measures, greatly reducing the burden and misjudgment rate of manual monitoring.

[0130] In the embodiment of the present invention, the target object is located in each frame, and the position of the target object in the image is recorded. As time goes by, the position points of the target object form a motion trajectory, including:

[0131] In each frame, the location of the target object is determined and a series of bounding boxes are output. Each bounding box corresponds to a detected target object. The bounding box marks the location of the object in the image, the category of the object, and the confidence of the detection.

[0132] Determine a preset confidence threshold, and obtain a bounding box with a confidence greater than the confidence threshold to obtain a filtered bounding box;

[0133] Extract the center point coordinates of the filtered bounding box, where the center point coordinates represent the position of the target object in the image frame;

[0134] The coordinates of each center point are associated with the timestamp of the frame image, and as the video frames are processed continuously, a series of target object position points associated with the timestamp are obtained;

[0135] The target object position points are connected in sequence according to the order of timestamps to form the motion trajectory of the target object.

[0136] In an embodiment of the present invention, by determining the position of the target object in each frame of the image and outputting a bounding box, the specific position of the target object in the image can be accurately marked. By setting a confidence threshold and filtering bounding boxes above the threshold, low-confidence detection results can be effectively removed, thereby improving the accuracy and reliability of target positioning. By extracting the coordinates of the center point of the filtered bounding box and associating it with the timestamp, the position change of the target object in each frame can be accurately tracked. As time goes by, these position points are connected in sequence to form a clear and accurate motion trajectory. Through the study of the motion trajectory, key information such as the speed, acceleration, and direction of movement of the target object can be extracted, thereby revealing its behavior pattern and intention.

[0137] In the embodiment of the present invention, for two adjacent frames of images, the displacement of the object on the image plane is calculated according to the position change of the target object in the image, including:

[0138] In the first frame image, the target object is located and the coordinates of the center point of the target object's bounding box are determined. The coordinates of the center point are (x1, y1), where x1 is the horizontal coordinate and y1 is the vertical coordinate;

[0139] In the second frame image, the target object is located and the center point coordinates of the target object's bounding box are determined. The center point coordinates are (x2, y2);

[0140] The change △x of the horizontal and vertical coordinates is calculated by subtracting the horizontal coordinate x1 of the first frame from the horizontal coordinate x2 of the second frame;

[0141] The change △y of the ordinate is calculated by subtracting the ordinate y1 of the first frame from the ordinate y2 of the second frame;

[0142] According to the changes in the horizontal and vertical coordinates and the vertical coordinates, the displacement (△x, △y) of the target object on the image plane is determined.

[0143] In an embodiment of the present invention, by directly comparing the coordinates of the center point of the target object's bounding box in two adjacent frames, the displacement of the target object on the image plane can be accurately calculated. By calculating each pair of adjacent frames, the continuous motion state of the target object can be efficiently tracked. Since only the calculation of two adjacent frames is involved, this method can provide the displacement information of the target object in real time. Compared with more complex motion estimation algorithms, this method has a small amount of calculation and can reduce the calculation burden of the system while ensuring a certain accuracy. This method does not depend on a specific scene or target object type, and can be widely used in various monitoring scenes to perform displacement calculations on different types of target objects.

[0144] For example, suppose you are processing a traffic surveillance video. In the first frame, a car is successfully located using an object detection algorithm (such as YOLO) and a bounding box is obtained. The center point of this bounding box is at (x1, y1), for example (300, 200). This means that in the first frame, the center of the car is approximately 300 pixels on the horizontal axis and 200 pixels on the vertical axis. When the next frame is processed, the object detection algorithm is used again to locate the same car and a new bounding box is obtained. The center point of this new bounding box is at (x2, y2), let's say (310, 210).

[0145] Now, we need to calculate the displacement of the car between the two frames. First, calculate the change in the horizontal coordinate △x:

[0146] △x=x2-x1=310-300=10 pixels; Next, calculate the change in the ordinate △y:

[0147] △y=y2-y1=210-200=10 pixels; determine the displacement of the target object on the image plane, so from the first frame to the second frame, the car moved 10 pixels to the right (△x=10) and 10 pixels down (△y=10) on the image plane. Therefore, the total displacement of the car between these two frames can be expressed as a vector (△x, △y), that is, (10, 10).

[0148] As the video continues to play and more frames are processed, the above process can be repeated to calculate the displacement of the car for each pair of adjacent frames. These displacement vectors are connected in sequence to form the motion trajectory of the car in the surveillance video. This process can help understand important information such as the vehicle's movement pattern, speed, and direction changes. Through this case, we can see that through continuous target detection and displacement calculation, the motion trajectory of the target object in the video can be accurately tracked and recorded.

[0149] In an embodiment of the present invention, extracting key features from the motion trajectory and speed data and determining the behavior pattern of the target object includes:

[0150] Extract key features from motion trajectories;

[0151] Screen key features to identify those related to the target object’s behavior patterns;

[0152] Cluster similar motion trajectories and speed features into the same group and perform clustering to obtain clustering results;

[0153] Based on the clustering results, the target object's motion trajectory is divided into different behavior pattern categories;

[0154] According to the clustering results, a behavior pattern label is assigned to each category of motion trajectory; the key features include the total length of the trajectory, the average curvature, the maximum curvature, and the number of turning points.

[0155] In an embodiment of the present invention, by extracting key features in the motion trajectory, such as the total length of the trajectory, the average curvature, the maximum curvature, the number of turning points, etc., the behavior pattern of the target object can be more accurately identified and understood. The key features are screened to determine the features most relevant to the behavior pattern of the target object, which improves the efficiency and accuracy of data analysis. By removing redundant and irrelevant features, you can focus more on those features that have an important impact on the classification of behavior patterns. Through cluster analysis, similar motion trajectories and speed features are clustered into the same group, which helps to discover and understand different behavior patterns. The clustering results can clearly show the different behavior categories of the target object. According to the clustering results, a behavior pattern label is assigned to each category of motion trajectory.

[0156] In another preferred embodiment of the present invention, extracting key features from the motion trajectory includes:

[0157] The total length of the trajectory is L, ,in,( , ) is a point on the trajectory, is the number of trajectory points.

[0158] Mean curvature , ,in, is at point ( , ) can be calculated from three adjacent points; the maximum curvature , ;

[0159] Calculate the correlation between features to screen key features, where the correlation calculation formula is:

[0160] ;in, is the total length of the trajectory of the ith sample, is the average of the total length of all sample trajectories, is at point ( , ), is the average of the mean curvatures of all samples, and are the deviations of the total trajectory length and mean curvature from their respective means, and n is the number of samples.

[0161] Use the elbow rule to determine the number of clusters K to be formed, and randomly select K data points from the data set as the initial cluster centers; for each data point in the data set (feature vector of the trajectory), calculate its Euclidean distance from each cluster center; assign each data point to the nearest cluster center to form a preliminary clustering result; for each cluster, calculate the mean of all data points in it, and this mean will become the new cluster center. Use the new cluster center to repeat the steps of assigning data points and recalculating cluster centers. This process will iterate until the preset maximum number of iterations is reached; when the iteration stops, each data point will be assigned to a final cluster center, representing a specific behavior pattern category; according to the characteristics of each cluster (such as average trajectory shape, speed change, etc.), assign them descriptive behavior pattern labels. For example, if the trajectories in a cluster are all straight lines and the speed is stable, they can be labeled as "straight and uniform speed driving". Output the behavior pattern category to which each trajectory belongs and its corresponding behavior pattern label.

[0162] In an embodiment of the present invention, by extracting key features such as the total length, average curvature and maximum curvature of the trajectory, the shape and dynamic changes of the trajectory can be intuitively reflected; by calculating the correlation between the features, the key features can be screened, redundant features can be removed, and the features that contribute most to the recognition of behavioral patterns can be retained. This helps to improve the accuracy and efficiency of clustering and reduce the waste of computing resources. Using the elbow rule to determine the number of clusters K can make the clustering results more consistent with the actual distribution of the data; clustering analysis of the data can cluster similar trajectories together to form different behavioral pattern categories. Since the cluster center is obtained through iterative calculation, the clustering result has a high accuracy.

[0163] In the embodiment of the present invention, judging whether the behavior of the target object conforms to the normal mode or the abnormal mode according to the judgment threshold includes:

[0164] Obtain the current judgment threshold from the dynamic adjustment module;

[0165] The target object's behavior data is compared with the judgment threshold. If all the target object's behavior data are ≤ the corresponding judgment threshold range, the target object's behavior is considered to be in a normal state. If any of the target object's behavior data is > the corresponding judgment threshold range, the target object's behavior is considered to be abnormal.

[0166] In an embodiment of the present invention, by comparing the behavior data of the target object with the dynamically adjusted judgment threshold in real time, the system can detect abnormal behavior of the target object in a timely and accurate manner. Since the judgment threshold is dynamically adjusted, the method can flexibly adjust the judgment criteria according to different environments and situations. By reasonably setting and adjusting the judgment threshold, the method can effectively reduce false alarms and missed alarms. This improves the accuracy and reliability of the monitoring system and reduces the workload of operators. The method enables the monitoring system to automatically classify the behavior of the target object as normal or abnormal based on the judgment threshold, improves the intelligence level of the system, and reduces the need for manual intervention.

[0167] For example, a large shopping mall installed an intelligent monitoring system to improve safety. The system can monitor the behavior of customers and employees in the shopping mall in real time and identify abnormal behavior through dynamically adjusted judgment thresholds.

[0168] Implementation steps:

[0169] The shopping mall has installed high-definition cameras in various key areas to capture the dynamic behaviors of customers and employees. The video data captured by the cameras are pre-processed and converted into analyzable behavioral data, such as movement speed, activity range, and dwell time.

[0170] The dynamic adjustment module dynamically adjusts the judgment threshold according to historical data and real-time conditions. For example, during peak shopping hours, when there are dense crowds, people's movement speed and activity range may increase appropriately, and the judgment threshold will be adjusted accordingly. Judgment thresholds include but are not limited to movement speed thresholds, activity range thresholds, and stay time thresholds.

[0171] The monitoring system obtains the behavior data of the target object (customer or employee) in real time. The behavior data of the target object is compared with the judgment threshold at the current moment. If all the behavior data of the target object (such as movement speed, activity range, and residence time) are within the judgment threshold range set by the dynamic adjustment module, its behavior is considered to be in line with the normal mode. If any behavior data of the target object exceeds the corresponding judgment threshold range, such as too fast movement speed, abnormal expansion of activity range, or too long residence time in a certain area, it is determined that the behavior of the target object is abnormal. Once abnormal behavior is detected, the system immediately triggers an alarm and notifies the security personnel through the shopping center's security system. The security personnel respond quickly, confirm the abnormal situation through the monitoring video, and take necessary measures, such as asking questions, stopping improper behavior, or calling the police. Through this intelligent monitoring system, the shopping center can detect abnormal behavior in real time and accurately, and respond in a timely manner, thereby effectively improving the safety of the shopping center.

[0172] In the embodiment of the present invention, when the data analysis module detects that the behavior pattern of the target object matches the preset abnormal behavior pattern, the early warning mechanism is triggered to generate and send early warning information, including:

[0173] The data analysis module monitors the behavior pattern of the target object, and compares the current behavior with the patterns in the preset abnormal behavior pattern library. If it is detected to be similar to the preset abnormal behavior pattern, the similarity score is calculated to evaluate the matching degree. If the similarity is greater than the preset threshold, the current behavior of the target object is considered abnormal;

[0174] When the behavior is confirmed to be abnormal, the data analysis module will trigger the early warning mechanism, which includes:

[0175] Record all relevant information about abnormal events, including timestamps, target object status, and behavior data;

[0176] Automatically generate an alert message that includes a description of the abnormal behavior, the time and location of the occurrence;

[0177] Sent to relevant personnel via communication channels.

[0178] In the embodiment of the present invention, by real-time monitoring of the behavior pattern of the target object and comparing it with the preset abnormal behavior pattern, the system can detect abnormal behavior and trigger an early warning in the first place. Using and similarity evaluation, the system can accurately determine whether the behavior of the target object matches the preset abnormal behavior pattern. This improves the accuracy of anomaly detection and reduces the possibility of false positives and false negatives. The early warning mechanism automatically records all relevant information of the abnormal event, including timestamps, the state of the target object, and behavior data.

[0179] In an embodiment of the present invention, a large shopping mall has deployed a customer abnormal behavior warning system to improve customer experience and safety. The system is designed to monitor customer behavior patterns in real time through a data analysis module. Once abnormal behavior is detected, such as theft, dispute, sudden fall, etc., the warning mechanism will be automatically triggered to notify security personnel in time for handling.

[0180] Implementation steps:

[0181] High-definition cameras and sensor networks are deployed in the shopping mall to capture and record customer behavior data in real time. Based on historical surveillance videos, security personnel's experience, and known abnormal behavior cases, a preset abnormal behavior pattern library is established, including theft, disputes, fights, sudden falls, etc. The data analysis module continuously receives data streams from cameras and sensors to analyze customer behavior. The customer's current behavior is compared with the preset abnormal behavior pattern library to find potential behavior matches.

[0182] If an action similar to an abnormal behavior pattern in the library is found, the system will calculate a similarity score and compare it with the preset threshold. When the similarity score exceeds the set threshold, the system determines that the customer's behavior is abnormal. Once the abnormal behavior is confirmed, the data analysis module will activate the early warning mechanism. The system automatically records key information such as the exact time when the abnormal behavior occurred, the specific location of the customer, and the characteristics of the behavior. Based on the recorded information, the system automatically generates an early warning message, detailing the nature, time and location of the abnormal behavior. This early warning message will be immediately sent to nearby security personnel through the shopping mall's internal communication system, such as intercoms, mobile phone APP push, etc. After receiving the early warning message, the security personnel will quickly go to the location where the abnormality occurred to intervene. Depending on the situation on the scene, the security personnel will take appropriate measures, such as stopping the conflict, providing medical assistance or contacting the police. In actual operation, the system successfully detected and warned of multiple theft attempts, conflicts between customers, and sudden health problems, ensuring the safety of customers and employees.

[0183] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An AI intelligent monitoring and management system, characterized in that: include: A data acquisition module is used to capture video data of the monitored area in real time through a camera; The data processing module is used to decode the video data, convert the compressed video data into an image frame sequence, preprocess each frame of the image, and use a deep learning model to perform target detection on the preprocessed image frame to identify the target object in the image; A dynamic adjustment module, used to dynamically determine the judgment threshold according to the real-time behavior characteristics of the target object and a preset dynamic adjustment factor; A data analysis module is used to analyze the movement trajectory, speed and behavior pattern of the target object, predict the behavior of the target object, and determine whether the behavior of the target object conforms to a normal mode or an abnormal mode based on a judgment threshold; The early warning module is used to trigger the early warning mechanism and generate and send early warning information when the data analysis module detects that the behavior pattern of the target object matches the preset abnormal behavior pattern; A control module is used to automatically initiate corresponding emergency response measures according to the level of the warning information; Decode the video data, convert the compressed video data into an image frame sequence, and pre-process each frame of the image, including: Reading a compressed video file or data stream, and inputting the compressed video data into a decoder for decoding operation to obtain original image block data; According to the frame sequence information obtained during the decoding process, the original image block data is rearranged in a playback order to form a continuous image frame sequence; For each image frame formed after decoding, perform color space conversion; After the color space conversion is completed, a blank image of the same size as the original image frame is created. Starting from the upper left corner of the image, each pixel of the image frame is traversed. The size of the blank image is the same as that of the original image frame and is used to store subsequent edge intensity information. For each pixel in the image, the total edge strength and edge direction are calculated according to its edge strength in the horizontal direction and the vertical direction to obtain the edge information of each pixel in the image; The calculated edge strength value is set as the pixel value of the output image at the corresponding position, and all pixels in the image frame are traversed and processed until the entire image frame is processed, and finally the filtered image frame is output; According to the real-time behavior characteristics of the target object and the preset dynamic adjustment factor, the judgment threshold is dynamically determined, including: Identify key behavioral characteristics, including speed, acceleration, and steering frequency; Set a base threshold for each key behavioral trait; For each key behavior feature, a dynamic adjustment amount is calculated according to the current value of the key behavior feature and the basic threshold, as well as a preset dynamic adjustment factor; According to the importance of each behavior feature, the corresponding weight is determined, and the corresponding weight is used to perform weighted summation on each dynamic adjustment amount to obtain a total dynamic adjustment value; Add the total dynamic adjustment value to the basic threshold to obtain a new judgment threshold; Analyze the target object's motion trajectory, speed, and behavior pattern, and predict its behavior, including: Locate the target object in each frame and record the position of the target object in the image. As time goes by, the position points of the target object form a motion trajectory. For two adjacent frames of images, the displacement of the object on the image plane is calculated according to the position change of the target object in the image; Convert displacement to velocity according to the frame rate of the image; Extract key features from motion trajectory and velocity data and determine the behavior patterns of target objects; Predict the future behavior of the target object based on its current state; The target object is located in each frame and its position in the image is recorded. As time goes by, the target object position points form a motion trajectory, including: In each frame, the location of the target object is determined and a series of bounding boxes are output. Each bounding box corresponds to a detected target object. The bounding box marks the location of the object in the image, the category of the object, and the confidence of the detection. Determine a preset confidence threshold, and obtain a bounding box with a confidence greater than the confidence threshold to obtain a filtered bounding box; Extract the center point coordinates of the filtered bounding box, where the center point coordinates represent the position of the target object in the image frame; The coordinates of each center point are associated with the timestamp of the frame image, and as the video frames are processed continuously, a series of target object position points associated with the timestamp are obtained; Connect the target object's position points in sequence according to the order of timestamps to form the target object's motion trajectory; For two adjacent frames of images, the displacement of the object on the image plane is calculated according to the position change of the target object in the image, including: In the first frame image, the target object is located and the coordinates of the center point of the target object's bounding box are determined. The coordinates of the center point are (x1, y1), where x1 is the horizontal coordinate and y1 is the vertical coordinate; In the second frame image, the target object is located and the center point coordinates of the target object's bounding box are determined. The center point coordinates are (x2, y2); The change △x of the horizontal and vertical coordinates is calculated by subtracting the horizontal coordinate x1 of the first frame from the horizontal coordinate x2 of the second frame; The change △y of the ordinate is calculated by subtracting the ordinate y1 of the first frame from the ordinate y2 of the second frame; Determine the displacement (△x, △y) of the target object on the image plane according to the change in the horizontal and vertical coordinates and the change in the vertical coordinate; Extract key features from motion trajectory and velocity data and determine the behavior patterns of target objects, including: Extract key features from motion trajectories; Screen key features to identify those related to the target object’s behavior patterns; Cluster similar motion trajectories and speed features into the same group and perform clustering to obtain clustering results; Based on the clustering results, the target object's motion trajectory is divided into different behavior pattern categories; According to the clustering results, a behavior pattern label is assigned to each category of motion trajectory; The key features include the total length, average curvature, maximum curvature, and number of turning points of the trajectory; Based on the judgment threshold, determine whether the behavior of the target object conforms to the normal mode or the abnormal mode, including: Obtain the current judgment threshold from the dynamic adjustment module; The target object's behavior data is compared with the judgment threshold. If all the target object's behavior data are ≤ the corresponding judgment threshold range, the target object's behavior is considered to be normal. If any of the target object's behavior data is > the corresponding judgment threshold range, the target object's behavior is considered to be abnormal. When the data analysis module detects that the behavior pattern of the target object matches the preset abnormal behavior pattern, the early warning mechanism is triggered to generate and send early warning information, including: The data analysis module monitors the behavior pattern of the target object, and compares the current behavior with the patterns in the preset abnormal behavior pattern library. If it is detected to be similar to the preset abnormal behavior pattern, the similarity score is calculated to evaluate the matching degree. If the similarity is greater than the preset threshold, the current behavior of the target object is considered abnormal; When the behavior is confirmed to be abnormal, the data analysis module will trigger the early warning mechanism, which includes: Record all relevant information about abnormal events, including timestamps, target object status, and behavior data; Automatically generate an alert message that includes a description of the abnormal behavior, the time and location of the occurrence; Sent to relevant personnel via communication channels.

Images