Pan / tilt recognition and capture method and system based on abnormal scenarios

Abstract

The present invention discloses a pan-tilt recognition and capture method and system based on abnormal scenarios, which relates to the technical field of intelligent devices; the video stream is subjected to frame extraction processing at a first preset period to obtain an initial image set; the target background and target image are determined according to the initial image set, and if the target image exists, the target image is subjected to background separation according to the target background to obtain a foreground image; the foreground image is subjected to initial key point extraction to obtain an initial key point set, and object recognition is performed according to the initial key point set, and if the object is recognized, the target key point is extracted from the object to obtain a target key point set; the object is tracked in real time according to the target key point set until the object cannot be recognized in the foreground area. The target background and target image are determined according to the image obtained by frame extraction, which avoids the direct use of a fixed background, improves the accuracy of subsequent object recognition, and then improves the accuracy and effectiveness of target recognition and capture by performing key point recognition on the recognized object.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent devices, and in particular relates to a pan / tilt recognition and capture method and system based on abnormal scenarios. Background Art

[0002] With the rapid development of computer technology and multimedia technology, video surveillance systems have been widely used in various industries and fields. Monitoring in many places no longer relies on manual intervention, but is completed through automated video surveillance systems. Video surveillance systems usually rely on pan-tilt cameras to achieve comprehensive coverage of the monitored area and can flexibly track and capture targets, thereby effectively protecting public safety, preventing potential risks, and avoiding accidents.

[0003] However, an important problem faced by PTZ cameras in real-time monitoring is the dynamic changes in the background. The background in reality is not fixed, but is affected by many factors and changes with time, environment and other external factors. For example, the ambient lighting conditions may change at any time, especially in different time periods such as the alternation of day and night or weather changes such as cloudy, sunny and rainy days. The scene captured by the camera will change significantly. These dynamic changes make it difficult for traditional static background modeling methods to cope with, which in turn affects the accurate separation of the detection target and the background, thereby greatly reducing the accuracy of subsequent target recognition and capture. Summary of the invention

[0004] The purpose of the present invention is to solve the problem that dynamic changes make it difficult for traditional static background modeling methods to cope with, which in turn affects the accurate separation of the detection target and the background, thereby greatly reducing the accuracy of subsequent target recognition and capture, and proposes a pan-tilt recognition and capture method and system based on abnormal scenarios.

[0005] In a first aspect of the present invention, a pan / tilt recognition and capture method based on abnormal scenarios is first proposed, the method comprising:

[0006] Acquire a video stream of a target area, and perform frame extraction processing on the video stream at a first preset period to obtain an initial image set;

[0007] Determine a target background and a target image according to the initial image set, and if the target image exists, perform background separation on the target image according to the target background to obtain a foreground image;

[0008] Extracting initial key points from the foreground image to obtain an initial key point set, performing object recognition based on the initial key point set, and if an object is recognized, extracting target key points from the object to obtain a target key point set;

[0009] The object is tracked in real time according to the target key point set until the foreground image cannot recognize the object.

[0010] Optionally, determining the target background and the target image according to the initial image set includes:

[0011] Step 1: for the initial image set, the initial image set is cut into a preset number to obtain a first image set; the first image set sorts the images in a time sequence of acquisition;

[0012] Step 2: Segment the target contrast image with a preset grid to obtain a first sub-image set, and for each sub-image in the first sub-image set, calculate the pixel average value and pixel variance value in the sub-image; each sub-image in the first sub-image set is numbered; the target contrast image is any one in the first image set;

[0013] Step 3: Obtain a target background image, and calculate the pixel average value difference and pixel variance value difference between the target background image and each sub-image in the target comparison image according to the serial number to obtain a first difference and a second difference;

[0014] Step 4: traverse the first difference and the second difference corresponding to each sub-image in the target background image and the target contrast image;

[0015] Step 5: If the first difference value corresponding to each sub-image is within the first threshold interval and the second difference value is within the second threshold interval, then the next image of the target contrast image is updated as the target contrast image, and the above steps 2 to 4 are repeated until the target contrast image is updated to the last image in the first image set, and the target background image is kept as the target background;

[0016] Step 6: If there is a sub-image whose first difference value is not within the first threshold interval but whose second difference value is within the second threshold interval, then the next image of the target contrast image is updated as the target contrast image, and the above steps 2 to 4 are repeated until the target contrast image is updated as the last image in the first image set. If there is no sub-image whose first difference value is not within the first threshold interval and whose second difference value is not within the second threshold interval after the target contrast image is updated, then the last image in the first image set is updated as the target background.

[0017] Step 7: If there is a sub-image whose corresponding second difference value is not within the second threshold interval, the target background image is kept as the target background, and the current target contrast image is recorded as the target image.

[0018] Optionally, performing background separation on the target image according to the target background to obtain a foreground image includes:

[0019] Substituting the target background and the target image into a Gaussian mixture model to obtain a first foreground pixel;

[0020] Performing differential calculation on each pixel point of the target image and the target background to obtain a target differential image;

[0021] Performing threshold judgment on the differential image according to a preset pixel threshold to obtain a second foreground pixel, and performing regional superposition on the first foreground pixel and the second foreground pixel to obtain a foreground mask image;

[0022] A foreground image is obtained by performing a closed operation on the foreground mask image and then performing a connected domain analysis.

[0023] Optionally, extracting target key points from the object to obtain a target key point set includes:

[0024] For the initial key point set corresponding to the object, fast corner point detection is performed on each initial key point in the initial key point set to obtain a response value of each initial key point;

[0025] Segmenting the initial key point set corresponding to the object according to the first preset interval to obtain a plurality of sub-initial key point set intervals;

[0026] For each sub-initial key point set interval, the target key points in the sub-initial key point set interval are determined according to the response value, and all target key points are obtained to obtain the target key point set.

[0027] Optionally, tracking the object in real time according to the target key point set includes:

[0028] For each target key point in the target key point set, the target key point is tracked in real time by an optical flow method to obtain a motion trajectory;

[0029] The object is captured and tracked according to the motion trajectories of all target key points.

[0030] In a second aspect of the present invention, a pan / tilt recognition and capture system based on abnormal scenarios is proposed, the system comprising:

[0031] A frame extraction module is used to obtain a video stream of a target area, and to perform frame extraction processing on the video stream at a first preset period to obtain an initial image set;

[0032] A background separation module is used to determine a target background and a target image according to the initial image set, and if the target image exists, perform background separation on the target image according to the target background to obtain a foreground image;

[0033] A key point extraction module is used to extract initial key points from the foreground image to obtain an initial key point set, perform object recognition based on the initial key point set, and if an object is recognized, extract target key points from the object to obtain a target key point set;

[0034] The tracking module is used to track the object in real time according to the target key point set until the foreground image cannot recognize the object.

[0035] Optionally, the background separation module includes:

[0036] A capture module, for the initial image set, captures the initial image set by a preset number to obtain a first image set; the first image set sorts the images in a time sequence of acquisition;

[0037] An image segmentation module, which segments the target contrast image into a first sub-image set by using a preset grid, and calculates the pixel average value and pixel variance value in each sub-image in the first sub-image set; each sub-image in the first sub-image set is numbered; and the target contrast image is any one of the first image set;

[0038] A difference calculation module is used to obtain a target background image, and to calculate a pixel average value difference and a pixel variance value difference between the target background image and each sub-image in the target comparison image according to the serial number to obtain a first difference and a second difference;

[0039] A sub-image traversal module, traversing the first difference and the second difference corresponding to each sub-image in the target background image and the target contrast image;

[0040] A first updating module, if the first difference value corresponding to each sub-image is within the first threshold interval and the second difference value is within the second threshold interval, then the next image of the target contrast image is updated as the target contrast image, and the image segmentation module, the difference calculation module and the sub-image traversal module are repeated until the target contrast image is updated to the last image in the first image set, and the target background image is kept as the target background;

[0041] A second updating module, if there is a sub-image whose first difference value is not within the first threshold interval but the second difference value is within the second threshold interval, then the next image of the target contrast image is updated as the target contrast image, and the image segmentation module, the difference calculation module and the sub-image traversal module are repeated until the target contrast image is updated to the last image in the first image set, and if there is no sub-image whose first difference value is not within the first threshold interval and the second difference value is not within the second threshold interval after the target contrast image is updated, then the last image in the first image set is updated as the target background;

[0042] The third updating module keeps the target background image as the target background if the second difference value corresponding to the sub-image is not within the second threshold interval, and records the current target contrast image as the target image.

[0043] Optionally, the background separation module further includes:

[0044] A first foreground pixel extraction module, used for substituting the target background and the target image into a Gaussian mixture model to obtain a first foreground pixel;

[0045] A target difference image generation module, used for performing a difference calculation on each pixel point of the target image and the target background to obtain a target difference image;

[0046] A pixel region superposition module, configured to perform a threshold judgment on the differential image according to a preset pixel threshold to obtain a second foreground pixel, and perform region superposition on the first foreground pixel and the second foreground pixel to obtain a foreground mask image;

[0047] The foreground image generation module is used to perform a closed operation on the foreground mask image and then perform a connected domain analysis to obtain a foreground image.

[0048] Optionally, the key point extraction module includes:

[0049] A response value generating module, for performing fast corner point detection on each initial key point in the initial key point set corresponding to the object to obtain a response value of each initial key point;

[0050] An initial key point partitioning module, used for partitioning the initial key point set corresponding to the object according to a first preset interval to obtain a plurality of sub-initial key point set intervals;

[0051] The target key point determination module is used to determine the target key points in each sub-initial key point set interval according to the response value, and obtain all the target key points to obtain the target key point set.

[0052] Optionally, the tracking module includes:

[0053] A motion trajectory determination module is used to obtain a motion trajectory by real-time tracking of each target key point in the target key point set using an optical flow method;

[0054] The capture and tracking module is used to capture and track the object according to the motion trajectory of all target key points.

[0055] Beneficial effects of the present invention:

[0056] The present invention proposes a pan-tilt recognition and capture method based on abnormal scenarios, which obtains a video stream of a target area, performs frame extraction processing on the video stream at a first preset period to obtain an initial image set; determines the target background and target image according to the initial image set, and if the target image exists, performs background separation on the target image according to the target background to obtain a foreground image; extracts initial key points from the foreground image to obtain an initial key point set, and performs object recognition according to the initial key point set, and if the object is recognized, extracts target key points from the object to obtain a target key point set; and tracks the object in real time according to the target key point set until the foreground image cannot recognize the object. Data processing is reduced by performing frame extraction processing on the captured video, and target background determination and target image determination are performed according to the image obtained by frame extraction, avoiding the direct use of a fixed background, improving the accuracy of subsequent object recognition, and then performing key point recognition on the recognized object to achieve real-time updating and tracking of the object, thereby improving the accuracy and effectiveness of target recognition and capture. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The present invention will be further described below in conjunction with the accompanying drawings.

[0058] Figure 1 A flowchart of a pan / tilt recognition and capture method based on abnormal scenarios is provided for an embodiment of the present invention;

[0059] Figure 2 A framework diagram of a pan / tilt recognition and capture system based on abnormal scenarios is provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the present invention, the description of "first", "second", etc. is only used for descriptive purposes, and cannot be understood as indicating or implying its relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" can explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0061] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of the present invention.

[0062] The embodiment of the present invention provides a pan / tilt recognition and capture method based on abnormal scenarios. Figure 1 , Figure 1 A flowchart of a pan / tilt recognition and capture method based on abnormal scenarios provided by an embodiment of the present invention. The method comprises the following steps:

[0063] S101, obtaining a video stream of a target area, and performing frame extraction processing on the video stream at a first preset period to obtain an initial image set;

[0064] S102, determining a target background and a target image according to the initial image set, and if a target image exists, performing background separation on the target image according to the target background to obtain a foreground image;

[0065] S103, extracting initial key points from the foreground image to obtain an initial key point set, performing object recognition based on the initial key point set, and if an object is recognized, extracting target key points from the object to obtain a target key point set;

[0066] S104, tracking the object in real time according to the target key point set until the foreground image cannot recognize the object.

[0067] A pan-tilt recognition and capture method based on abnormal scenarios is provided based on an embodiment of the present invention. Data processing is reduced by performing frame extraction on the captured video. Target background and target image are determined based on the image obtained by frame extraction, thereby avoiding direct use of a fixed background, thereby improving the accuracy of subsequent object recognition. Key point recognition is then performed on the recognized object, thereby achieving real-time updating and tracking of the object, thereby improving the accuracy and effectiveness of target recognition and capture.

[0068] In one implementation, a first preset period is determined by a technician; if no object is identified, continuous object identification is performed, and an alarm instruction is sent to relevant personnel when a preset first time period is reached; in an area where the target area does not require detection, an alarm instruction is sent to relevant personnel when the duration of the detected object exceeds a preset second time period; the preset first time period and the preset second time period are determined by a technician, and when no object can be identified in the foreground image, the gimbal returns to the initial angle of capturing the target area.

[0069] In one implementation, Harris corner detection is performed on the extracted foreground to obtain an initial key point set, and feature point matching is performed on the key points in the initial key point set through SIFT. When no match is successful, the image is recaptured and the above steps are performed until the feature point matching is successful to determine the target object. The target object determined by the feature point matching is determined by matching the existing database.

[0070] In one implementation, through frame extraction processing and the establishment of a preliminary image set, key frame information can be extracted, thereby reducing redundant data and improving the accuracy of target recognition; by separating the background to obtain the foreground image, background interference can be effectively removed, focusing on the extraction of the target object, making target recognition more stable and reliable in complex environments.

[0071] In one implementation, determining the target background and the target image through the initial image set can ensure that when the target image is detected, the background is updated, thereby improving the accuracy of subsequent object recognition.

[0072] In one implementation, initial key point extraction can help the system obtain the target's feature points and then perform object recognition. In order to determine the target's key points, by real-time tracking of the target's key point set, the system can continuously track the target in the video stream while reducing key point detection and improving tracking speed.

[0073] In one embodiment, determining the target background and the target image according to the initial image set includes:

[0074] Step 1: for an initial image set, the initial image set is cut into a preset number to obtain a first image set; the first image set sorts the images according to the time sequence of acquisition;

[0075] Step 2: Segment the target contrast image with a preset grid to obtain a first sub-image set, and for each sub-image in the first sub-image set, calculate the pixel average value and pixel variance value in the sub-image; each sub-image in the first sub-image set is numbered; the target contrast image is any one in the first image set;

[0076] Step 3: Obtain the target background image, and calculate the pixel average value difference and pixel variance value difference between the target background image and each sub-image in the target comparison image according to the serial number to obtain a first difference and a second difference;

[0077] Step 4: traverse the first difference and the second difference corresponding to each sub-image in the target background image and the target comparison image;

[0078] Step 5: If the first difference value corresponding to each sub-image is within the first threshold interval and the second difference value is within the second threshold interval, the next image of the target contrast image is updated as the target contrast image, and the above steps 2 to 4 are repeated until the target contrast image is updated to the last image in the first image set, and the target background image is kept as the target background;

[0079] Step 6: If there is a sub-image whose first difference value is not within the first threshold interval but whose second difference value is within the second threshold interval, the next image of the target contrast image is updated as the target contrast image, and the above steps 2 to 4 are repeated until the target contrast image is updated to the last image in the first image set. If there is no sub-image whose first difference value is not within the first threshold interval and whose second difference value is not within the second threshold interval after the target contrast image is updated, the last image in the first image set is updated as the target background;

[0080] Step 7: If there is a sub-image whose corresponding second difference value is not within the second threshold interval, the target background image is kept as the target background, and the current target contrast image is recorded as the target image.

[0081] In one implementation, the target image is gridded, the image is divided into multiple sub-regions, and the pixel mean and variance of each sub-region are calculated. This can improve the efficiency of image processing, reduce the amount of image data processed each time, and more accurately analyze different regions in the image.

[0082] In one implementation, the capture rule is that the images in the initial image set are sorted in time series, starting from the currently acquired image, a preset number of images are captured in reverse order to obtain the first image set, and the first image set is still sorted in the acquisition time series; the preset number is determined by the technician.

[0083] In one implementation, the preset grid is determined by a technician and is usually set to a 32 by 32 pixel grid; the sub-images in the image grid set are numbered; the target background image is a background image determined in advance by the above scheme, and the sub-images of the target background image are also numbered by the above method; the pixel average value difference and the pixel variance value difference of each sub-image in the target background image and the target comparison image are calculated according to the number to obtain a first difference and a second difference. Specifically, for the sub-image of the target background image (image A) and the sub-image of the target comparison image (image B) with the same number, the first difference is obtained by subtracting the pixel average value of image A from the pixel average value of image B, and the second difference is obtained by subtracting the pixel variance value of image A from the pixel variance value of image B.

[0084] In one implementation, the first threshold interval and the second threshold interval are determined by a technician. When the first difference is within the first threshold interval and the second difference is within the second threshold interval, it means that the current picture has not changed. This is also the case when all images in the first image set are compared. This means that the current picture has not changed and no background update is required. If the first difference of the sub-image is not within the first threshold interval, but the second difference is within the second threshold interval, the target comparison image is updated. This means that the average pixel value of the sub-image has changed significantly, but the pixel variance has not changed much, indicating that the entire sub-area has changed synchronously. This may be due to natural changes such as lighting and weather. At this time, the last image in the first image set is updated as the target background. If the second difference of the sub-image is not within the second threshold interval, it means that the structure or texture between the target and the background has changed greatly, which usually indicates that the target image has significant movement or change. At this time, it is judged that an object appears in the image and the background image needs to remain unchanged.

[0085] In one implementation, by calculating the pixel mean difference and pixel variance difference between the background image and each sub-region in the current comparison image, the subtle differences between the background and the target can be accurately captured, thereby more accurately distinguishing the foreground and background, helping to determine whether the changes in the image come from the target object and reducing misidentification.

[0086] In one embodiment, performing background separation on the target image according to the target background to obtain a foreground image includes:

[0087] Substituting the target background and the target image into the Gaussian mixture model to obtain the first foreground pixel;

[0088] Perform differential calculation on each pixel point of the target image and the target background to obtain a target differential image;

[0089] Performing threshold judgment on the differential image according to a preset pixel threshold to obtain a second foreground pixel, and performing regional superposition on the first foreground pixel and the second foreground pixel to obtain a foreground mask image;

[0090] The foreground image is obtained by performing a closed operation on the foreground mask image and then performing a connected domain analysis.

[0091] In one implementation, by modeling the color distribution of each pixel, dynamic targets and static backgrounds can be effectively distinguished. The Gaussian mixture model can handle complex scenes, including lighting changes, dynamic backgrounds, etc. Therefore, the first foreground pixel can effectively mark those pixels that are different from the background, thereby distinguishing the foreground target; the preset pixel threshold is determined by the technician.

[0092] In one implementation, the contrast between the background and the target can be further enhanced by calculating the difference between the target image and the target background. The difference operation can highlight the changed areas in the image and identify those pixels with significantly different pixels, so that dynamic targets can be effectively extracted even when there are slight changes in the background.

[0093] In one implementation, by regionally superimposing the first foreground pixel and the second foreground pixel, the advantages of different extraction methods can be comprehensively considered, the extraction quality of the foreground pixel is further improved, and the accuracy is ensured; the closing operation is a common operation in morphological image processing, which is used to eliminate small noise points or incomplete foreground images in the foreground image and fill the holes of the target. A smoother and more coherent foreground image can be obtained through the closing operation system, and then the connected domain analysis can be performed to effectively identify the various target areas in the image, ensuring the integrity of the target area and the accuracy of the segmentation.

[0094] In one embodiment, extracting target key points from an object to obtain a target key point set includes:

[0095] For the initial key point set corresponding to the object, a fast corner point detection is performed on each initial key point in the initial key point set to obtain a response value of each initial key point;

[0096] Segmenting the initial key point set corresponding to the object according to the first preset interval to obtain a plurality of sub-initial key point set intervals;

[0097] For each sub-initial key point set interval, the target key points in the sub-initial key point set interval are determined according to the response value, and all target key points are obtained to obtain the target key point set.

[0098] In one implementation, fast corner point detection can be used to quickly identify important local feature points in an image, which can efficiently extract important information from the image, reduce the amount of calculation, and quickly locate potential target areas; the initial key point set is divided into multiple subsets, so that separate regional analysis is performed on the initial subset in each area, which helps to eliminate some irrelevant noise, thereby improving the accuracy and efficiency of target detection; the response value is related to the change in pixel brightness, reflecting the texture change of the area around the point.

[0099] In one implementation, the first preset interval is determined by a technician, usually an interval of 16 by 16; segmentation is to segment the image of the area where the object is located, and determine the target key point in the sub-initial key point set interval according to the response value, specifically, sorting the response values ​​in the interval from large to small, and obtaining the first key point as the target key point.

[0100] In one embodiment, real-time tracking of an object based on a target key point set includes:

[0101] For each target key point in the target key point set, the motion trajectory is obtained by real-time tracking of the target key point using the optical flow method;

[0102] The object is captured and tracked according to the motion trajectory of all target key points.

[0103] In one implementation, the optical flow method is a method for estimating the motion of an object by calculating the motion of pixels in an image sequence. The optical flow method can estimate the motion trajectory of each target key point in the video stream in real time; by tracking each target key point, the precise motion trajectory of each key point can be obtained. The point-to-point tracking method is more detailed than the rough tracking of the overall target, and can accurately capture the detailed motion of the target, especially when the target moves quickly or deforms, it can still stably track the changes of the target.

[0104] In one implementation method, the movement speed (velocity vector) and acceleration of each key point are calculated, and the movement speed and acceleration of each key point are brought into a linear model to simulate the overall movement trajectory of the target. Based on the simulated overall movement trajectory of the target, the acceleration model is used to predict the future position of the target.

[0105] In one implementation, by identifying the motion trajectories of all target key points, the system can capture and track the target more accurately. By analyzing the motion trajectories of the key points, it can infer the overall movement direction, speed, and acceleration of the target, thereby making predictions in advance when the target may be occluded, mismatched, or the movement speed changes, ensuring that the target is always effectively tracked.

[0106] Based on the same inventive concept, the embodiment of the present invention also provides a pan / tilt recognition and capture system based on abnormal scenarios. Figure 2 , Figure 2 A framework diagram of a pan / tilt recognition and capture system based on abnormal scenarios provided by an embodiment of the present invention includes:

[0107] A frame extraction module is used to obtain a video stream of a target area, and to extract frames from the video stream at a first preset period to obtain an initial image set;

[0108] A background separation module is used to determine the target background and the target image according to the initial image set. If the target image exists, the target image is separated from the background according to the target background to obtain a foreground image.

[0109] A key point extraction module is used to extract initial key points from the foreground image to obtain an initial key point set, perform object recognition based on the initial key point set, and if an object is recognized, extract target key points from the object to obtain a target key point set;

[0110] The tracking module is used to track the object in real time according to the target key point set until the foreground image cannot recognize the object.

[0111] A pan-tilt recognition and capture system based on abnormal scenarios is provided based on an embodiment of the present invention. Data processing is reduced by performing frame extraction on the captured video. Target background and target image are determined based on the images obtained by frame extraction, thereby avoiding direct use of a fixed background and improving the accuracy of subsequent object recognition. Key points of the identified objects are then identified to achieve real-time updating and tracking of the line, thereby improving the accuracy and effectiveness of target recognition and capture.

[0112] In one embodiment, the background separation module includes:

[0113] The interception module intercepts the initial image set by a preset number to obtain a first image set; the first image set sorts the images according to the time sequence of acquisition;

[0114] An image segmentation module, which segments the target contrast image with a preset grid to obtain a first sub-image set, and for each sub-image in the first sub-image set, calculates a pixel average value and a pixel variance value in the sub-image; each sub-image in the first sub-image set is numbered; the target contrast image is any one in the first image set;

[0115] The difference calculation module obtains the target background image, and calculates the pixel average value difference and the pixel variance value difference between the target background image and each sub-image in the target comparison image according to the serial number to obtain the first difference and the second difference;

[0116] A sub-image traversal module, traversing the first difference and the second difference corresponding to each sub-image in the target background image and the target comparison image;

[0117] A first updating module, if the first difference value corresponding to each sub-image is within the first threshold interval and the second difference value is within the second threshold interval, then the next image of the target contrast image is updated to the target contrast image, and the image segmentation module, the difference calculation module and the sub-image traversal module are repeated until the target contrast image is updated to the last image in the first image set, and the target background image is kept as the target background;

[0118] A second updating module, if there is a sub-image whose first difference value is not within the first threshold interval but whose second difference value is within the second threshold interval, then the next image of the target contrast image is updated to the target contrast image, and the image segmentation module, the difference calculation module and the sub-image traversal module are repeated until the target contrast image is updated to the last image in the first image set, and if there is no sub-image whose first difference value is not within the first threshold interval and whose second difference value is not within the second threshold interval after the target contrast image is updated, then the last image in the first image set is updated to the target background;

[0119] The third updating module keeps the target background image as the target background if the second difference value corresponding to the sub-image is not within the second threshold interval, and records the current target contrast image as the target image.

[0120] In one embodiment, the background separation module further includes:

[0121] A first foreground pixel extraction module, used for substituting the target background and the target image into a Gaussian mixture model to obtain a first foreground pixel;

[0122] A target difference image generation module is used to perform a difference calculation on each pixel point between the target image and the target background to obtain a target difference image;

[0123] A pixel region superposition module is used to perform a threshold judgment on the differential image according to a preset pixel threshold to obtain a second foreground pixel, and to perform region superposition on the first foreground pixel and the second foreground pixel to obtain a foreground mask image;

[0124] The foreground image generation module is used to perform a closed operation on the foreground mask image and then perform a connected domain analysis to obtain the foreground image.

[0125] In one embodiment, the key point extraction module includes:

[0126] A response value generation module is used to perform fast corner point detection on each initial key point in the initial key point set corresponding to the object to obtain a response value of each initial key point;

[0127] An initial key point partitioning module, used to partition the initial key point set corresponding to the object according to the first preset interval to obtain a plurality of sub-initial key point set intervals;

[0128] The target key point determination module is used to determine the target key points in each sub-initial key point set interval according to the response value, and obtain all the target key points to obtain the target key point set.

[0129] In one embodiment, the tracking module includes:

[0130] A motion trajectory determination module is used to obtain a motion trajectory by real-time tracking of each target key point in the target key point set using an optical flow method;

[0131] The capture and tracking module is used to capture and track the object according to the motion trajectory of all target key points.

[0132] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A pan / tilt recognition and capture method based on abnormal scenarios, characterized in that: The method comprises: Acquire a video stream of a target area, and perform frame extraction processing on the video stream at a first preset period to obtain an initial image set; Determine a target background and a target image according to the initial image set, and if the target image exists, perform background separation on the target image according to the target background to obtain a foreground image; Extracting initial key points from the foreground image to obtain an initial key point set, performing object recognition based on the initial key point set, and if an object is recognized, extracting target key points from the object to obtain a target key point set; Tracking the object in real time according to the target key point set until the foreground image cannot recognize the object; Determining the target background and the target image according to the initial image set includes: Step 1: for the initial image set, the initial image set is cut into a preset number to obtain a first image set; the first image set sorts the images in a time sequence of acquisition; Step 2: Segment the target contrast image with a preset grid to obtain a first sub-image set, and for each sub-image in the first sub-image set, calculate the pixel average value and pixel variance value in the sub-image; each sub-image in the first sub-image set is numbered; the target contrast image is any one in the first image set; Step 3: Obtain a target background image, and calculate the pixel average value difference and pixel variance value difference between the target background image and each sub-image in the target comparison image according to the serial number to obtain a first difference and a second difference; Step 4: traverse the first difference and the second difference corresponding to each sub-image in the target background image and the target contrast image; Step 5: If the first difference value corresponding to each sub-image is within the first threshold interval and the second difference value is within the second threshold interval, then the next image of the target contrast image is updated as the target contrast image, and the above steps 2 to 4 are repeated until the target contrast image is updated to the last image in the first image set, and the target background image is kept as the target background; Step 6: If there is a sub-image whose first difference value is not within the first threshold interval but whose second difference value is within the second threshold interval, then the next image of the target contrast image is updated as the target contrast image, and the above steps 2 to 4 are repeated until the target contrast image is updated as the last image in the first image set. If there is no sub-image whose first difference value is not within the first threshold interval and whose second difference value is not within the second threshold interval after the target contrast image is updated, then the last image in the first image set is updated as the target background. Step 7: If there is a sub-image whose corresponding second difference value is not within the second threshold interval, the target background image is kept as the target background, and the current target contrast image is recorded as the target image.

2. The method for pan / tilt recognition and capture based on abnormal scenarios according to claim 1, characterized in that: Performing background separation on the target image according to the target background to obtain a foreground image includes: Substituting the target background and the target image into a Gaussian mixture model to obtain a first foreground pixel; Performing differential calculation on each pixel point of the target image and the target background to obtain a target differential image; Performing threshold judgment on the differential image according to a preset pixel threshold to obtain a second foreground pixel, and performing regional superposition on the first foreground pixel and the second foreground pixel to obtain a foreground mask image; A foreground image is obtained by performing a closed operation on the foreground mask image and then performing a connected domain analysis.

3. The method for pan / tilt recognition and capture based on abnormal scenarios according to claim 1, characterized in that: Extracting target key points from the object to obtain a target key point set includes: For the initial key point set corresponding to the object, fast corner point detection is performed on each initial key point in the initial key point set to obtain a response value of each initial key point; Segmenting the initial key point set corresponding to the object according to the first preset interval to obtain a plurality of sub-initial key point set intervals; For each sub-initial key point set interval, the target key points in the sub-initial key point set interval are determined according to the response value, and all target key points are obtained to obtain the target key point set.

4. The method for pan / tilt recognition and capture based on abnormal scenarios according to claim 1, characterized in that: Tracking the object in real time according to the target key point set includes: For each target key point in the target key point set, the target key point is tracked in real time by an optical flow method to obtain a motion trajectory; The object is captured and tracked according to the motion trajectories of all target key points.

5. A pan / tilt recognition and capture system based on abnormal scenarios, characterized in that: The system comprises: A frame extraction module is used to obtain a video stream of a target area, and to perform frame extraction processing on the video stream at a first preset period to obtain an initial image set; A background separation module is used to determine a target background and a target image according to the initial image set, and if the target image exists, perform background separation on the target image according to the target background to obtain a foreground image; A key point extraction module is used to extract initial key points from the foreground image to obtain an initial key point set, perform object recognition based on the initial key point set, and if an object is recognized, extract target key points from the object to obtain a target key point set; A tracking module, used for tracking the object in real time according to the target key point set until the foreground image cannot recognize the object; The background separation module comprises: A capture module, for the initial image set, captures the initial image set by a preset number to obtain a first image set; the first image set sorts the images in a time sequence of acquisition; An image segmentation module, which segments the target contrast image into a first sub-image set by using a preset grid, and calculates the pixel average value and pixel variance value in each sub-image in the first sub-image set; each sub-image in the first sub-image set is numbered; and the target contrast image is any one of the first image set; A difference calculation module is used to obtain a target background image, and to calculate a pixel average value difference and a pixel variance value difference between the target background image and each sub-image in the target comparison image according to the serial number to obtain a first difference and a second difference; A sub-image traversal module, traversing the first difference and the second difference corresponding to each sub-image in the target background image and the target contrast image; A first updating module, if the first difference value corresponding to each sub-image is within the first threshold interval and the second difference value is within the second threshold interval, then the next image of the target contrast image is updated as the target contrast image, and the image segmentation module, the difference calculation module and the sub-image traversal module are repeated until the target contrast image is updated to the last image in the first image set, and the target background image is kept as the target background; A second updating module, if there is a sub-image whose first difference value is not within the first threshold interval but the second difference value is within the second threshold interval, then the next image of the target contrast image is updated as the target contrast image, and the image segmentation module, the difference calculation module and the sub-image traversal module are repeated until the target contrast image is updated to the last image in the first image set, and if there is no sub-image whose first difference value is not within the first threshold interval and the second difference value is not within the second threshold interval after the target contrast image is updated, then the last image in the first image set is updated as the target background; The third updating module keeps the target background image as the target background if the second difference value corresponding to the sub-image is not within the second threshold interval, and records the current target contrast image as the target image.

6. The PTZ recognition and capture system based on abnormal scenarios according to claim 5, characterized in that: The background separation module also includes: A first foreground pixel extraction module, used for substituting the target background and the target image into a Gaussian mixture model to obtain a first foreground pixel; A target difference image generation module, used for performing a difference calculation on each pixel point of the target image and the target background to obtain a target difference image; A pixel region superposition module, configured to perform a threshold judgment on the differential image according to a preset pixel threshold to obtain a second foreground pixel, and perform region superposition on the first foreground pixel and the second foreground pixel to obtain a foreground mask image; The foreground image generation module is used to perform a closed operation on the foreground mask image and then perform a connected domain analysis to obtain a foreground image.

7. The PTZ recognition and capture system based on abnormal scenarios according to claim 5, characterized in that: The key point extraction module comprises: A response value generating module, for performing fast corner point detection on each initial key point in the initial key point set corresponding to the object to obtain a response value of each initial key point; An initial key point partitioning module, used for partitioning the initial key point set corresponding to the object according to a first preset interval to obtain a plurality of sub-initial key point set intervals; The target key point determination module is used to determine the target key points in each sub-initial key point set interval according to the response value, and obtain all the target key points to obtain the target key point set.

8. The PTZ recognition and capture system based on abnormal scenarios according to claim 5, characterized in that: The tracking module includes: A motion trajectory determination module is used to obtain a motion trajectory by real-time tracking of each target key point in the target key point set using an optical flow method; The capture and tracking module is used to capture and track the object according to the motion trajectory of all target key points.

Images