Intelligent cigarette end trajectory analysis method and analysis system based on video monitoring

Abstract

The invention provides an intelligent cigarette end track analysis method and analysis system based on video monitoring, and relates to the technical field of video analysis, and the method comprises the steps: obtaining a monitoring video stream, and carrying out the step-by-step screening of the monitoring video stream based on a multi-stage filtering mechanism, and obtaining a video clip containing a cigarette end; in the video clip containing the cigarette end, performing continuous inter-frame tracking on the cigarette end by using a target tracking method to obtain a cigarette end movement track; analyzing a correlation degree between the cigarette end movement track and a fire occurrence time point, and obtaining an anomaly detection result based on the correlation degree; according to the method, the intelligent level of fire monitoring is greatly improved, and automatic identification, tracking and analysis of tiny targets such as cigarette ends are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of video analysis, and in particular to an intelligent cigarette butt trajectory analysis method and analysis system based on video monitoring. Background Art

[0002] In the field of fire accident investigation and analysis, accurate identification of the cause of fire is crucial to preventing future fires, maintaining public safety and determining the attribution of responsibility. Among them, fires caused by objects thrown from high places (such as cigarette butts thrown from high buildings) or cigarette butts thrown on the ground are common. Such fires pose great challenges to investigations because of their strong concealment and difficulty in capturing evidence.

[0003] Traditionally, fire investigators rely on surveillance videos as a key source of evidence to track abnormal behavior before a fire, especially the discarding of cigarette butts. However, capturing the trajectory of cigarette butts in surveillance videos faces multiple technical challenges: First, cigarette butts are small in size and often appear as tiny and fast-moving objects in video footage, making them easily overlooked; second, the time interval from the discarding of cigarette butts to the ignition of a fire is long, which may span from minutes to hours, requiring investigators to check the video frame by frame for a long time and with high concentration, which is a lot of work and inefficient; third, existing surveillance systems and image processing technologies lack efficient recognition and tracking algorithms for tiny moving objects, making it difficult to automate the investigation process.

[0004] Although video analysis technology has made significant progress in the current market, customized solutions for specific scenarios (such as tracking cigarette butts before a fire) are still lacking. Existing video surveillance systems mainly focus on large-scale monitoring, face recognition, abnormal behavior detection and other functions, and have limited ability to accurately identify and track small, atypical targets (such as cigarette butts) for a long time, which cannot meet the actual needs of fire investigation work.

[0005] Therefore, it is necessary to provide an intelligent cigarette butt trajectory analysis method and analysis system based on video surveillance to solve the above technical problems. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides an intelligent cigarette butt trajectory analysis method and analysis system based on video surveillance, which greatly improves the intelligence level of fire monitoring and realizes automatic recognition, tracking and analysis of subtle targets such as cigarette butts.

[0007] The present invention provides an intelligent cigarette butt trajectory analysis method based on video monitoring, the analysis method comprising the following steps:

[0008] Acquire a surveillance video stream, and filter the surveillance video stream level by level based on a multi-level filtering mechanism to obtain a video clip containing a cigarette butt;

[0009] In the video clip containing the cigarette butt, the cigarette butt is tracked between consecutive frames using a target tracking method to obtain a movement trajectory of the cigarette butt;

[0010] Analyzing the correlation between the cigarette butt movement trajectory and the fire occurrence time point, and obtaining an abnormality detection result based on the correlation degree;

[0011] The cigarette butt movement trajectory and abnormal detection results are visualized.

[0012] Preferably, after acquiring the surveillance video stream and before screening step by step, the method further includes:

[0013] The acquired surveillance video stream is preprocessed, wherein the preprocessing includes denoising, contrast enhancement and image stabilization.

[0014] Preferably, the acquiring of the surveillance video stream and screening the surveillance video stream level by level based on a multi-level filtering mechanism to obtain a video clip containing a cigarette butt comprises:

[0015] Using a background difference method to perform foreground separation on the pre-processed surveillance video stream to obtain a foreground area;

[0016] In the foreground area, primary screening is performed on candidate objects based on primary conditions to exclude candidate objects that do not meet the morphological characteristics of cigarette butts, wherein the candidate objects represent objects in the foreground area, and the primary conditions are shape thresholds and size thresholds;

[0017] Apply the pre-built color model to perform secondary screening on the candidates after primary screening, and exclude the candidates that do not meet the color characteristics of cigarette butts;

[0018] The candidate objects after the secondary screening are finally confirmed by using a machine learning model, and video clips containing cigarette butts are intercepted from the surveillance video stream according to the result of the final confirmation.

[0019] Preferably, the target tracking method uses any one of a Kalman filter, a particle filter or an optical flow method to track the movement trajectory of the cigarette butt.

[0020] Preferably, in the video clip containing the cigarette butts, tracking the cigarette butts between consecutive frames using a target tracking method to obtain the movement trajectory of the cigarette butts includes:

[0021] estimating the position of the cigarette butt between consecutive frames in the video clip containing the cigarette butt, and determining the position coordinates of the cigarette butt in each frame;

[0022] Calculating the moving direction and speed of the cigarette butt according to the change of the position coordinates;

[0023] The cigarette butt motion trajectory is generated by combining the changes in the position coordinates, movement direction and speed of the cigarette butt between consecutive frames.

[0024] Preferably, analyzing the correlation between the cigarette butt movement trajectory and the fire occurrence time point, and obtaining an abnormality detection result based on the correlation degree, includes:

[0025] Extracting key features of the cigarette butt motion trajectory, wherein the key features include the moving speed, moving direction, dwell time and trajectory shape of the cigarette butt;

[0026] The key features are matched with the predefined cigarette butt behavior pattern before the fire, and the matching degree is calculated, wherein the calculation formula of the matching degree is:

[0027]

[0028] Among them, M i The matching degree between the i-th predefined cigarette butt behavior pattern before the fire and the observed cigarette butt motion trajectory, i represents the number of predefined cigarette butt behavior patterns before the fire, S i represents the Euclidean distance between the i-th predefined cigarette butt behavior pattern before the fire and the observed cigarette butt motion trajectory. The calculation formula of the Euclidean distance is:

[0029]

[0030] Among them, S i represents the matching degree between the i-th predefined cigarette butt behavior pattern before the fire and the current cigarette butt motion trajectory, f j represents the jth key feature of the cigarette butt trajectory, f ij represents the jth key feature of the i-th predefined cigarette butt behavior pattern before fire;

[0031] Determining the degree of correlation between the cigarette butt movement trajectory and the fire occurrence time point according to the matching degree;

[0032] When the correlation degree exceeds a preset threshold, it is determined to be abnormal and an abnormality detection result is generated.

[0033] Preferably, the visual display of the cigarette butt movement trajectory and abnormality detection result includes:

[0034] On the monitoring video stream, marking the movement trajectory of the cigarette butt in a graphical manner, wherein the graphical manner includes one or a combination of lines, arrows or animation trajectories;

[0035] The movement trajectory of cigarette butts determined to be abnormal is highlighted, and the abnormality detection result is displayed on the screen of the monitoring video, wherein the abnormality detection result includes the abnormality type, correlation degree and occurrence time.

[0036] The present invention provides an intelligent cigarette butt trajectory analysis system based on video surveillance, which is used to perform an intelligent cigarette butt trajectory analysis method based on video surveillance. The analysis system includes:

[0037] A video clip acquisition module is used to acquire a surveillance video stream and filter the surveillance video stream level by level based on a multi-level filtering mechanism to obtain a video clip containing a cigarette butt;

[0038] A cigarette butt motion trajectory acquisition module is used to track the cigarette butt between consecutive frames using a target tracking method in the video clip containing the cigarette butt to obtain the cigarette butt motion trajectory;

[0039] An abnormality judgment module, used to analyze the correlation between the movement trajectory of the cigarette butt and the time point of the fire occurrence, and obtain an abnormality detection result based on the correlation degree;

[0040] A visualization module is used to visualize the movement trajectory of the cigarette butt and the abnormality detection result.

[0041] Compared with the related art, the intelligent cigarette butt trajectory analysis method and analysis system based on video surveillance provided by the present invention has the following beneficial effects:

[0042] The present invention screens the monitoring video stream step by step through a multi-level filtering mechanism to accurately extract video clips containing cigarette butts; utilizes advanced target tracking technology to accurately track cigarette butts between consecutive frames to generate cigarette butt motion trajectories; analyzes the correlation between the cigarette butt motion trajectory and the time point of fire occurrence to achieve real-time monitoring and early warning of potential fire risks; finally, through visual display technology, the cigarette butt motion trajectory and abnormal detection results are intuitively displayed to users, providing strong support for fire prevention. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A flow chart of an intelligent cigarette butt trajectory analysis method based on video surveillance provided by the present invention;

[0044] Figure 2 A module structure diagram of an intelligent cigarette butt trajectory analysis system based on video surveillance provided by the present invention. DETAILED DESCRIPTION

[0045] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only the parts related to the present invention, rather than all structures, are shown in the accompanying drawings. In addition, the embodiments of the present invention and the features in the embodiments may be combined with each other without conflict.

[0046] It should also be noted that, for ease of description, only the part relevant to the present invention but not all the content is shown in the accompanying drawings. It should be mentioned before discussing the exemplary embodiments in more detail that some exemplary embodiments are described as the processing or method described as a flow chart. Although the flow chart describes each operation (or step) as being processed sequentially, many operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. When its operation is completed, the processing can be terminated, but it can also have additional steps not included in the accompanying drawings. The processing can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0047] Embodiment 1

[0048] The present invention provides an intelligent cigarette butt trajectory analysis method based on video surveillance, referring to Figure 1 As shown, the analysis method comprises the following steps:

[0049] S1: Obtain a surveillance video stream, and filter the surveillance video stream level by level based on a multi-level filtering mechanism to obtain a video clip containing a cigarette butt.

[0050] Specifically, in step S1, after acquiring the monitoring video stream, the following steps are also included:

[0051] The acquired surveillance video stream is preprocessed, wherein the preprocessing includes denoising, contrast enhancement and image stabilization.

[0052] In this embodiment, denoising can remove noise interference in the video by digital signal processing technology, including but not limited to using a median filtering method or a Gaussian filtering method to smooth the image and reduce the impact of random noise.

[0053] Contrast enhancement is to improve the contrast of the image through image enhancement technology, making targets such as cigarette butts more obvious and facilitating subsequent feature extraction. This can be achieved by using methods including but not limited to histogram equalization methods or local contrast enhancement methods.

[0054] In addition, since surveillance cameras may be affected by factors such as wind and vibration and cause slight shaking, image stabilization technology is needed to eliminate this shaking and keep the picture stable. This is specifically achieved through algorithms such as feature matching.

[0055] Specifically, step S1 includes the following steps:

[0056] S11: Using a background difference method to separate the foreground of the pre-processed surveillance video stream to obtain a foreground area.

[0057] In this embodiment, the background difference method is specifically: identifying the foreground by comparing the difference between the current frame and the background model, wherein the background model is derived based on historical frame statistics, and illustratively, a mixed Gaussian model is used to model the background, and then a threshold is set to distinguish the foreground and background.

[0058] S12: In the foreground area, primary screening is performed on candidate objects based on primary conditions to exclude candidate objects that do not meet the morphological characteristics of cigarette butts, wherein the candidate objects represent objects in the foreground area, and the primary conditions are shape thresholds and size thresholds.

[0059] In this embodiment, the main purpose of this step is to exclude those candidate objects that obviously do not meet the characteristics of cigarette butts by setting reasonable shape and size thresholds, thereby reducing false detection in subsequent processing. It is specifically divided into the following steps:

[0060] The Canny edge detection algorithm is used to extract the edges in the image. Canny edge detection is a multi-stage algorithm that uses a Gaussian filter to remove noise, calculates the gradient strength and direction, and then determines the edge through non-maximum suppression and double threshold detection. At the same time, appropriate high and low thresholds are selected to ensure that the edge of the cigarette butt can be detected without introducing too many noise edges.

[0061] Based on the detected edges, the contour search function in the OpenCV library is used to extract the contour. The contour is a set of points with continuous edges in the image. These points form a closed curve and represent the boundary of the object. At the same time, the minimum and maximum contour areas are usually set to filter out contours that are too small or too large, thereby eliminating noise.

[0062] The size screening is specifically to set a reasonable diameter range (for example, 3-6 pixels). If the equivalent circle diameter of the contour is within this range, it is considered to be a cigarette butt. For example, for each contour, its area can be calculated, and then the equivalent circle diameter can be calculated.

[0063] Shape screening is to determine whether the shape is close to a circle or an ellipse by calculating the aspect ratio of the contour. For example, for each contour, its circumscribed rectangle can be found, its width and height can be calculated, and then the aspect ratio can be calculated. If the aspect ratio is between 0.5 and 2, it is considered to be close to a circle or an ellipse.

[0064] S13: Apply the pre-built color model to perform secondary screening on the candidate objects after the primary screening, and exclude the candidate objects that do not meet the color characteristics of cigarette butts.

[0065] In this embodiment, this step further screens out candidate objects that match the characteristics of cigarette butts by color characteristics, and excludes objects that have the right shape and size but do not match the characteristics of cigarette butts by setting a reasonable color range. The specific process is as follows:

[0066] The HSV color space is selected as the color model. The HSV (Hue, Saturation, Value) color space is widely used in color recognition tasks because it is more robust to lighting changes. The Hue channel in the HSV space represents the type of color (such as red, green, etc.), the Saturation channel represents the color saturation, and the Value channel represents the brightness.

[0067] Define a reasonable color range: Cigarette butts usually appear red or orange when burning. Therefore, a reasonable HSV color range can be defined to match these colors; for example, the Hue value of red in HSV space is usually between 0° and 10° or 160° to 180°, and orange is between 11° and 25°.

[0068] Red range: (0, 100, 100) to (10, 255, 255) and (160, 100, 100) to (180, 255, 255)

[0069] Orange range: (11, 100, 100) to (25, 255, 255)

[0070] Using the color range defined above, create a mask in the HSV color space, i.e. a binary image where pixels whose colors match the defined range are marked as white (255) and the rest are marked as black (0). Apply the mask to the original image and only keep the pixels corresponding to the white areas in the mask, thus highlighting the areas that match the color characteristics of the cigarette butts.

[0071] The results obtained by color screening are combined with the previous results based on shape and size screening to further improve the detection accuracy.

[0072] S14: Using a machine learning model to perform final confirmation on the candidate objects after the secondary screening, and based on the final confirmation result, extracting a video clip containing a cigarette butt from the surveillance video stream.

[0073] In this embodiment, the purpose of this step is to conduct a final confirmation on the candidate objects that have been initially screened through a machine learning model to determine whether they are cigarette butts. The specific process is as follows:

[0074] Collect a large number of images containing cigarette butts and non-cigarette butts as training data. These images should cover various lighting conditions, angles, and backgrounds. Label the cigarette butt and non-cigarette butt areas in each image, usually using bounding boxes to mark the location of the cigarette butts. Increase the diversity of the data set by rotating, scaling, flipping, etc. to help the model generalize better.

[0075] Convolutional Neural Networks (CNNs) are chosen as the machine learning model. CNNs are well suited for image classification tasks because they can automatically learn features in images.

[0076] Select an appropriate CNN architecture, including but not limited to VGG, ResNet, Inception, and adjust the number of layers and parameters as needed. Use the cross-entropy loss function to train the classification model. Select Adam as the optimizer to speed up the training process. Then, adjust hyperparameters such as learning rate and batch size through cross-validation methods. Use the validation set to evaluate the performance of the model on unseen data and adjust the model to avoid overfitting. After training, deploy the model to process new image data.

[0077] S2: In the video clip containing the cigarette butt, the cigarette butt is tracked between consecutive frames using a target tracking method to obtain a movement trajectory of the cigarette butt.

[0078] Specifically, step S2 includes the following steps:

[0079] S21: estimating the position of the cigarette butt between consecutive frames in the video clip containing the cigarette butt, and determining the position coordinates of the cigarette butt in each frame.

[0080] In this embodiment, a target tracking algorithm, including but not limited to a Kalman filter, a particle filter or an optical flow method, is used to estimate the position of the cigarette butt between consecutive frames.

[0081] Among them, the optical flow method estimates the motion of the object by calculating the optical flow field of each point in the image sequence.

[0082] Kalman filter and particle filter can effectively deal with the uncertainty of the target.

[0083] Specific implementation:

[0084] First, the tracker is initialized. For the first appearance of a cigarette butt, the bounding box identified by the machine learning model in step S14 can be used as the initial position.

[0085] In subsequent frames, the position of the cigarette butt is updated using the selected tracking algorithm. For example, when the optical flow method is used, its movement can be estimated based on the displacement of the feature points of the cigarette butt between two adjacent frames.

[0086] In each frame, the position coordinates of the cigarette butt are recorded (the center coordinates are used to represent the position coordinates) for subsequent trajectory analysis.

[0087] S22: Calculate the moving direction and speed of the cigarette butt according to the change of the position coordinates.

[0088] In this embodiment, the moving direction and speed are calculated using the position coordinate changes of the cigarette butt between consecutive frames.

[0089] The moving direction can be estimated by the change vector of the position coordinates between adjacent frames.

[0090] The movement speed can be calculated by dividing the change distance of the position coordinates between adjacent frames by the frame rate.

[0091] Specific implementation:

[0092] For each frame, calculate the difference between the current position coordinates of the cigarette butt and the position coordinates of the previous frame.

[0093] Calculate the moving direction vector based on the difference.

[0094] Moving speed = moving distance / time interval (assuming the time interval for each frame is the same).

[0095] S23: Generate a cigarette butt motion trajectory by combining the change in the position coordinates, the moving direction and the speed of the cigarette butt between consecutive frames.

[0096] In this embodiment, the motion trajectory is a visual representation of the path of the cigarette butt moving over time.

[0097] Trajectories can be used to analyze the behavior patterns of cigarette butts.

[0098] Specific implementation:

[0099] Connect the position coordinates of the cigarette butts in each frame to form a trajectory.

[0100] Use, including but not limited to, lines or arrows to indicate the path and direction of movement of the cigarette butt.

[0101] The thickness or color of the track can change according to the moving speed of the cigarette butt to intuitively show the change in speed.

[0102] S3: Analyze the correlation between the cigarette butt movement trajectory and the fire occurrence time point, and obtain an abnormality detection result based on the correlation degree.

[0103] Specifically, step S3 includes the following steps:

[0104] S31: extracting key features from the movement trajectory of the cigarette butt, wherein the key features include the movement speed, movement direction, residence time and trajectory shape of the cigarette butt.

[0105] In this embodiment, in this step, a series of key features need to be extracted from the movement trajectory of the cigarette butt, which will be used for subsequent analysis and matching. The key features include but are not limited to the movement speed, movement direction, dwell time and trajectory shape of the cigarette butt.

[0106] Movement speed: Calculate the displacement of the cigarette butt in each frame and then calculate its average speed. This can be done by measuring the difference in the position coordinates of the cigarette butt between consecutive frames.

[0107] Moving direction: The moving direction is determined based on the displacement vector of the cigarette butt between consecutive frames.

[0108] Dwell time: Records the length of time a cigarette butt remains in a small area, which helps identify if the butt has been discarded or placed in a fixed location.

[0109] Trajectory shape: Analyze the path of the cigarette butts to see if there is a pattern, such as a straight line, curve or complex trajectory.

[0110] S32: Match the key feature with the predefined cigarette butt behavior pattern before the fire, and calculate the matching degree, wherein the calculation formula of the matching degree is:

[0111]

[0112] Among them, M i The matching degree between the i-th predefined cigarette butt behavior pattern before the fire and the observed cigarette butt motion trajectory, i represents the number of predefined cigarette butt behavior patterns before the fire, S i represents the Euclidean distance between the i-th predefined cigarette butt behavior pattern before the fire and the observed cigarette butt motion trajectory. The calculation formula of the Euclidean distance is:

[0113]

[0114] Among them, S i represents the matching degree between the i-th predefined cigarette butt behavior pattern before the fire and the current cigarette butt motion trajectory, f j represents the jth key feature of the cigarette butt trajectory, f ij Represents the jth key feature of the i-th predefined cigarette butt behavior pattern before fire.

[0115] In this embodiment, in this step, the extracted key features are matched with the predefined cigarette butt behavior patterns before fire to determine whether the behavior of the cigarette butt indicates a potential fire risk.

[0116] Predefined behavior patterns of cigarette butts before a fire: Based on previous data and experience, a set of typical behavior patterns of cigarette butts before a fire are defined, such as cigarette butts moving quickly on the ground or staying in a certain position for a long time.

[0117] Matching degree calculation: Euclidean distance is used to measure the similarity between the observed cigarette butt trajectory and the predefined pattern. The smaller the distance, the higher the matching degree.

[0118] S33: Determine the degree of correlation between the cigarette butt movement trajectory and the fire occurrence time point according to the matching degree.

[0119] In this embodiment, in this step, the correlation between the cigarette butt movement trajectory and the potential fire occurrence is determined based on the calculation result of the matching degree.

[0120] Degree of association: The degree of association between the trajectory of the cigarette butt and the occurrence of the fire is determined based on the degree of match. For example, if the degree of match between multiple key features and the behavior pattern of the cigarette butt before the fire is high, the degree of association is considered high.

[0121] S34: When the correlation degree exceeds a preset threshold, it is determined to be abnormal and an abnormality detection result is generated.

[0122] In this embodiment, this step is to determine whether to trigger an alarm based on the judgment result of the association degree and generate a corresponding abnormality detection result.

[0123] Abnormal determination: If the degree of correlation exceeds a preset threshold, abnormal behavior is considered to exist.

[0124] Anomaly detection results: Generate a report that includes the anomaly type (such as potential fire risk), the specific value of the correlation degree, the time when the abnormal behavior occurred, etc.

[0125] S4: Visually displaying the cigarette butt movement trajectory and abnormality detection results.

[0126] Specifically, step S4 includes the following steps:

[0127] S41: Marking the movement trajectory of the cigarette butt in a graphical manner on the monitoring video stream, wherein the graphical manner includes one or a combination of lines, arrows or animation trajectories.

[0128] In this embodiment, in step S41, the movement trajectory of the cigarette butt is graphically marked on the monitoring video stream. These graphical methods may include lines, arrows or animation trajectories, etc., in order to intuitively display the movement path and direction of the cigarette butt.

[0129] Line marking: Use lines of different colors to indicate the movement trajectory of cigarette butts, such as using red lines to indicate the movement path of cigarette butts. The thickness of the lines can be dynamically adjusted according to the movement speed of the cigarette butts. The lines are thicker when the movement speed is fast and thinner when the movement speed is slow.

[0130] Arrow marking: Place arrows at the location of cigarette butts to indicate the direction of movement of cigarette butts. The color of the arrows can be consistent with the color of the lines to facilitate the distinction of different cigarette butt trajectories.

[0131] Animation trajectory: While the video is playing, the movement trajectory of the cigarette butt is dynamically displayed so that the trajectory is updated in real time as the cigarette butt actually moves, which helps to intuitively understand the movement path and speed changes of the cigarette butt.

[0132] S42: highlighting the movement trajectory of the cigarette butt determined to be abnormal, and displaying the abnormality detection result on the screen of the monitoring video, wherein the abnormality detection result includes the abnormality type, correlation degree and occurrence time.

[0133] In this embodiment, in step S42, the movement trajectory of the cigarette butt determined to be abnormal is specially marked, and the abnormality detection result is displayed on the screen of the monitoring video.

[0134] For cigarette butt movement trajectories that are determined to be abnormal, special colors (such as yellow or orange) are used to highlight them to facilitate rapid identification of these abnormal trajectories.

[0135] Anomaly detection results display:

[0136] Abnormality type: clearly mark the reason for the abnormal movement trajectory of the cigarette butt, such as "suspected discarded cigarette butt caused the fire."

[0137] Correlation degree: Displays the correlation degree between the movement trajectory of cigarette butts and the time point of fire occurrence. It can be a percentage value or a level (such as low, medium, high).

[0138] Time of occurrence: Mark the specific time when the abnormal behavior occurs to facilitate subsequent tracing.

[0139] The working principle of the intelligent cigarette butt trajectory analysis method based on video monitoring provided by the present invention is as follows:

[0140] First, the system obtains the surveillance video stream as input.

[0141] In order to improve the accuracy and efficiency of subsequent processing, the acquired surveillance video stream is preprocessed, including denoising, contrast enhancement and image stabilization. These preprocessing steps help reduce interference factors in the video and make the cigarette butt target more prominent.

[0142] Secondly, the preprocessed video stream enters a multi-level filtering mechanism for screening step by step.

[0143] The background difference method is used to separate the foreground of the video stream and obtain the foreground area.

[0144] In the foreground area, the candidate objects are screened based on the primary conditions (shape threshold and size threshold) to exclude the candidate objects that do not meet the morphological characteristics of cigarette butts.

[0145] The pre-built color model is applied to perform secondary screening on the candidates after the primary screening to further exclude the candidates that do not meet the color characteristics of cigarette butts.

[0146] Finally, the machine learning model is used to make final confirmation on the candidates after the secondary screening, and the video clips containing cigarette butts are cut out from the video stream based on the confirmation results.

[0147] Next, in the video clip containing the cigarette butt, the target tracking method is used to track the cigarette butt between consecutive frames.

[0148] By estimating the position of the cigarette butt between consecutive frames, the position coordinates of the cigarette butt in each frame are determined.

[0149] According to the change of position coordinates, the moving direction and speed of the cigarette butt are calculated.

[0150] The cigarette butt motion trajectory is generated by combining the position coordinate changes, moving direction and speed of the cigarette butt between consecutive frames.

[0151] Analysis of the correlation between cigarette butt movement trajectory and fire:

[0152] Extract key features of the cigarette butt motion trajectory, including moving speed, moving direction, dwell time and trajectory shape.

[0153] These key features are matched with the predefined behavior patterns of cigarette butts before fire, and the matching degree is calculated.

[0154] The degree of correlation between the cigarette butt movement trajectory and the time point of fire occurrence is determined based on the matching degree.

[0155] When the correlation degree exceeds the preset threshold, it is determined to be abnormal and an abnormal detection result is generated.

[0156] Finally, the cigarette butt movement trajectory and anomaly detection results are visualized.

[0157] On the surveillance video stream, the movement trajectory of the cigarette butt is marked in a graphical manner (such as lines, arrows or animation trajectories).

[0158] The movement trajectory of cigarette butts judged to be abnormal is highlighted, and the abnormality detection results are displayed on the surveillance video screen, including the abnormality type, correlation degree and occurrence time.

[0159] In summary, the present invention realizes intelligent analysis of the movement trajectory of cigarette butts in surveillance videos, and performs correlation analysis with the time point of fire occurrence, and finally displays the analysis results in a visual way. This method provides a new technical means for fire early warning and monitoring, which helps to improve the accuracy and efficiency of fire prevention.

[0160] Embodiment 2

[0161] The present invention provides an intelligent cigarette butt trajectory analysis system based on video surveillance, which is used to perform an intelligent cigarette butt trajectory analysis method based on video surveillance. Figure 2 As shown, the analysis system comprises:

[0162] The video clip acquisition module 100 is used to acquire a surveillance video stream and filter the surveillance video stream level by level based on a multi-level filtering mechanism to obtain a video clip containing a cigarette butt.

[0163] The cigarette butt motion trajectory acquisition module 200 is used to track the cigarette butt between consecutive frames in the video clip containing the cigarette butt using a target tracking method to obtain the cigarette butt motion trajectory.

[0164] The abnormality judgment module 300 is used to analyze the correlation between the movement trajectory of the cigarette butt and the time point of the fire occurrence, and obtain the abnormality detection result based on the correlation degree.

[0165] The visualization module 100 is used to visualize the movement trajectory of the cigarette butt and the abnormality detection result.

[0166] The present application is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0167] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, the storage medium including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0168] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

Claims

1. An intelligent cigarette butt trajectory analysis method based on video surveillance, characterized in that: The analytical method comprises the following steps: Acquire a surveillance video stream, and filter the surveillance video stream level by level based on a multi-level filtering mechanism to obtain a video clip containing a cigarette butt; In the video clip containing the cigarette butt, the cigarette butt is tracked between consecutive frames using a target tracking method to obtain a movement trajectory of the cigarette butt; Analyzing the correlation between the cigarette butt movement trajectory and the fire occurrence time point, and obtaining an abnormality detection result based on the correlation degree; The cigarette butt movement trajectory and abnormal detection results are visualized.

2. According to the video surveillance-based intelligent cigarette butt trajectory analysis method of claim 1, it is characterized in that: After acquiring the surveillance video stream and before screening step by step, the method further includes: The acquired surveillance video stream is preprocessed, wherein the preprocessing includes denoising, contrast enhancement and image stabilization.

3. The intelligent cigarette butt trajectory analysis method based on video surveillance according to claim 2 is characterized in that: The step of acquiring the surveillance video stream and filtering the surveillance video stream level by level based on a multi-level filtering mechanism to obtain a video clip containing a cigarette butt includes: Using a background difference method to perform foreground separation on the pre-processed surveillance video stream to obtain a foreground area; In the foreground area, primary screening is performed on candidate objects based on primary conditions to exclude candidate objects that do not meet the morphological characteristics of cigarette butts, wherein the candidate objects represent objects in the foreground area, and the primary conditions are shape thresholds and size thresholds; Apply the pre-built color model to perform secondary screening on the candidates after primary screening, and exclude the candidates that do not meet the color characteristics of cigarette butts; The candidate objects after the secondary screening are finally confirmed by using a machine learning model, and video clips containing cigarette butts are intercepted from the surveillance video stream according to the result of the final confirmation.

4. The intelligent cigarette butt trajectory analysis method based on video surveillance according to claim 3 is characterized in that: The target tracking method adopts any one of Kalman filter, particle filter or optical flow method to realize the tracking of the movement trajectory of the cigarette butt.

5. The intelligent cigarette butt trajectory analysis method based on video surveillance according to claim 4 is characterized in that: In the video clip containing the cigarette butt, tracking the cigarette butt between consecutive frames using a target tracking method to obtain a movement trajectory of the cigarette butt includes: estimating the position of the cigarette butt between consecutive frames in the video clip containing the cigarette butt, and determining the position coordinates of the cigarette butt in each frame; Calculating the moving direction and speed of the cigarette butt according to the change of the position coordinates; The cigarette butt motion trajectory is generated by combining the changes in the position coordinates, movement direction and speed of the cigarette butt between consecutive frames.

6. The intelligent cigarette butt trajectory analysis method based on video surveillance according to claim 5 is characterized in that: The analyzing the correlation between the cigarette butt movement trajectory and the fire occurrence time point, and obtaining an abnormality detection result based on the correlation, includes: Extracting key features of the cigarette butt motion trajectory, wherein the key features include the movement speed, movement direction, residence time and trajectory shape of the cigarette butt; The key features are matched with the predefined cigarette butt behavior pattern before the fire, and the matching degree is calculated, wherein the calculation formula of the matching degree is: Among them, M i The matching degree between the i-th predefined cigarette butt behavior pattern before the fire and the observed cigarette butt motion trajectory, i represents the number of predefined cigarette butt behavior patterns before the fire, S i represents the Euclidean distance between the i-th predefined cigarette butt behavior pattern before the fire and the observed cigarette butt motion trajectory. The calculation formula of the Euclidean distance is: Among them, S i represents the matching degree between the i-th predefined cigarette butt behavior pattern before the fire and the current cigarette butt motion trajectory, f j represents the jth key feature of the cigarette butt trajectory, f ij represents the jth key feature of the i-th predefined cigarette butt behavior pattern before fire; Determining the degree of correlation between the cigarette butt movement trajectory and the fire occurrence time point according to the matching degree; When the correlation degree exceeds a preset threshold, it is determined to be abnormal and an abnormality detection result is generated.

7. The intelligent cigarette butt trajectory analysis method based on video surveillance according to claim 6 is characterized in that: The visual display of the cigarette butt movement trajectory and the abnormality detection result includes: On the monitoring video stream, marking the movement trajectory of the cigarette butt in a graphical manner, wherein the graphical manner includes one or more combinations of lines, arrows or animation trajectories; The movement trajectory of cigarette butts determined to be abnormal is highlighted, and the abnormality detection result is displayed on the screen of the monitoring video, wherein the abnormality detection result includes the abnormality type, correlation degree and occurrence time.

8. An intelligent cigarette butt trajectory analysis system based on video surveillance, used to execute an intelligent cigarette butt trajectory analysis method based on video surveillance as claimed in any one of claims 1 to 7, characterized in that: The analysis system comprises: A video clip acquisition module is used to acquire a surveillance video stream and filter the surveillance video stream level by level based on a multi-level filtering mechanism to obtain a video clip containing a cigarette butt; A cigarette butt motion trajectory acquisition module is used to track the cigarette butt between consecutive frames using a target tracking method in the video clip containing the cigarette butt to obtain the cigarette butt motion trajectory; An abnormality judgment module, used to analyze the correlation between the movement trajectory of the cigarette butt and the time point of the fire occurrence, and obtain an abnormality detection result based on the correlation degree; A visualization module is used to visualize the movement trajectory of the cigarette butt and the abnormality detection result.

Images