A video stream change detection method and system based on image registration

Abstract

The present invention relates to a video stream change detection method and system based on image registration, the method comprising: obtaining a reference image and a change image to be registered, and performing key point detection respectively; and matching the key points; using a random sampling consistency algorithm to purify the obtained key point pairs, and obtaining multiple inlier point sets and multiple homography matrices; determining the detection area according to all the inliers, and mapping the key points in the reference image to the change image according to the obtained homography matrix, and then comparing these paired key points to obtain the changed key points; calculating the maximum connected domain of the changed key points, so as to obtain the change frame; setting a buffer according to the context information of the change image to reduce false detection and missed detection. Compared with the prior art, the present invention solves the constraints of traditional homography estimation, expands the scope of key point matching, and improves the accuracy of homography estimation; at the same time, introduces a buffer method to reduce false detection and missed detection of video stream change detection.

Description

Technical Field

[0001] The present invention relates to the technical field of video stream change detection, and in particular to a video stream change detection method and system based on image registration. Background Art

[0002] The change detection task refers to the process of identifying the state differences between two or more images of the same scene taken at different times, lighting, viewpoints or under more complex conditions. Originating from the development of multi-temporal remote sensing (RS) data, synthetic aperture radar (SAR) data and medical images, change detection has achieved great success in the fields of remote sensing and medicine. At present, many deep learning methods have been proposed for remote sensing and medical change detection, such as STANet, ChangeNet and other twin network-based algorithms. However, due to the lack of a close-range change detection dataset, that is, a dataset containing close-range images taken under different viewpoints or lighting conditions, there is currently no deep learning change detection method for close-range images. In order to cope with the change detection of close-range images, the general approach is to first register the images to obtain aligned images, and then perform difference detection on the aligned images. Image registration can be divided into several steps: key point detection, key point matching, homography estimation and image transformation.

[0003] In recent years, deep learning technology has been used in various processes of image registration to replace traditional algorithms because it can obtain image depth features and representations from linear and nonlinear spaces through multi-layer networks. The paper "Superpoint: Self-supervised interest point detection and description" (DeTone D, Malisiewicz T, Rabinovich A. / / Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 2018: 224-236.) proposed the SuperPoint self-supervised full convolution model, which is robust to key point detection under illumination and seasonal changes. The input of SuperPoint is a full-size image, and it calculates the pixel-level key point position and key point descriptor in one forward propagation. SuperPoint consists of MagicPoint and homography adaptation module. MagicPoint is pre-trained on a synthetic geometric dataset, and the homography adaptation module enables it to achieve self-supervised training while improving detection repeatability.

[0004] The paper "Key.net: Keypoint detection by handcrafted and learned cnnfilters" (Barroso-Laguna A, Riba E, Ponsa D, et al. Proceedings of the IEEE / CVFInternational Conference on Computer Vision. 2019: 5836-5844.) proposed the Key.Net shallow multi-scale architecture model, which is a key point detector that combines traditional feature detection ideas and deep learning detection ideas. Traditional detection filters provide an "anchor structure" for deep learning filters, which locate, score and rank repeatable features. The multi-scale architecture ensures the invariance of Key.Net to scale changes.

[0005] Although key point detection technology has made great progress, traditional homography estimation ignores the importance of dealing with depth differences and moving objects in real-world applications, has more constraints on scenes, and cannot meet the homography requirements of complex scenes. For video stream data, image change detection-based methods ignore the contextual information of the video and cannot be directly used for video stream change detection. Summary of the invention

[0006] The purpose of the present invention is to provide a method and system for detecting video stream changes based on image registration in order to overcome the defects of the above-mentioned prior art that key points have a small coverage range and ignore the context information of the video.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] A video stream change detection method based on image registration, characterized in that it comprises the following steps:

[0009] S1: Obtain a reference image and a variation image to be registered, and perform key point detection on the reference image and the variation image respectively;

[0010] S2: Matching key points of the reference image and the variation image;

[0011] S3: Use a random sampling consensus algorithm to purify the key point pairs obtained in step S2 to obtain multiple internal point sets and multiple homography matrices;

[0012] S4: determining the detection area according to the inliers in the set of all inliers, and mapping the key points in the reference image to the change image according to the obtained homography matrix, and then comparing these pairs of key points to obtain the changed key points;

[0013] S5: Calculate the maximum connected domain of the changed key points obtained in step S4, so as to obtain a change box;

[0014] S6: Setting a buffer to store the change box according to the context information of the change graph, and displaying the change box if the number of consecutive appearances of the change box in the buffer reaches a preset number threshold.

[0015] Furthermore, the step S1 specifically includes:

[0016] S101: Set the maximum number of key points to be extracted, and jump to step S102;

[0017] S102: using a key point detection method to detect key points from the reference image and the variation image respectively, if the number of key points detected is greater than the maximum number of key points, jump to step S103, otherwise jump to S104;

[0018] S103: Sort the key points according to the strength of the detected key points, select the key points with the largest strength, and jump to step S104;

[0019] S104: Calculate the descriptor of the key point.

[0020] Furthermore, the step S2 specifically includes:

[0021] S201: Select a search algorithm, set the number of traversal times and distance threshold, and jump to step S202;

[0022] S202: According to the search algorithm and the number of traversals set in step S202, a fast nearest neighbor search matcher is constructed to match the key points of the reference image and the change image, and then the process goes to step S203;

[0023] S203: Filtering out false matches of the matched key points according to the distance threshold.

[0024] Furthermore, in step S3, when calculating the homography matrix using the random sampling consistency algorithm, inliers and outliers are obtained. If the proportion of the outliers in the key points is greater than a preset outlier proportion threshold, the homography matrix is ​​continued to be calculated based on the outliers to obtain multiple homography matrices.

[0025] Furthermore, the step S4 is specifically as follows:

[0026] The detection area is determined according to the inliers in the set of all inliers, and the key points are re-divided at fixed intervals within the detection area. For the re-divided key points, the K nearest neighbor classification algorithm is used to determine the category of the current key point based on the nearest several inliers, so as to select the corresponding homography matrix for mapping; finally, the descriptors of the paired key points in the reference image and the changed image are compared to obtain the changed key point pairs.

[0027] Furthermore, in step S5, a change frame is obtained according to the circumscribed rectangle of the maximum connected domain.

[0028] Furthermore, the step S6 is specifically as follows: a buffer is set to store the change box according to the context information of the change graph; if the number of consecutive appearances of the change box in the buffer reaches a preset appearance threshold, the change box is displayed; if the number of consecutive non-detection of the displayed change box reaches a preset non-appearance threshold, the change box is removed from the buffer.

[0029] Furthermore, the step S6 specifically includes:

[0030] For the change box detected in the current frame, if the buffer is empty, the change box is directly stored in the buffer and the number of occurrences is set to 1; if the buffer is not empty, the change boxes in the buffer are traversed to match the change boxes of the current frame. If the match is successful, the number of occurrences of the corresponding change box in the buffer is increased by 1. If the number of occurrences of the change box is greater than the preset number of occurrences threshold, the change box is displayed; the number of occurrences of the unmatched change box in the buffer is reduced by 1. If the number of occurrences of the change box in the buffer is less than 0, the corresponding change box is removed.

[0031] Furthermore, the change graph is an image frame in a video stream.

[0032] The present invention also provides a video stream change detection system based on image registration, comprising a memory and a processor, wherein the memory stores a computer program, and the processor calls the computer program to execute the steps of the above method.

[0033] Compared with the prior art, the present invention has the following advantages:

[0034] The significant advantage of the present invention is that it proposes a change detection method that combines key point detection and registration, which does not rely on massive data for training. At the same time, key point detection is performed by multiple methods, so that key point detection under changes in illumination and viewpoint can be handled. The present invention proposes a multi-homography matching method, which solves the constraints of traditional homography estimation, expands the scope of key point matching, and improves the accuracy of homography estimation. The present invention also designs a buffer to reduce missed detection and false detection in video stream change detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A schematic diagram of a flow chart of a video stream change detection method based on image registration provided in an embodiment of the present invention;

[0036] Figure 2 A flowchart of calculating a multi-homography matrix provided in an embodiment of the present invention;

[0037] Figure 3 A schematic diagram of a processing flow of a buffer zone provided in an embodiment of the present invention;

[0038] Figure 4 This is a diagram of a homography estimation experiment result under illumination changes provided in an embodiment of the present invention;

[0039] Figure 5 This is a diagram of a homography estimation experiment result under viewpoint change provided in an embodiment of the present invention;

[0040] Figure 6 This is a graph of change detection experiment results provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0042] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0044] Example 1

[0045] This embodiment provides a video stream change detection method based on image registration, comprising the following steps:

[0046] S1: Obtain a reference image and a variation image to be registered, and perform key point detection on the reference image and the variation image respectively;

[0047] S2: Matching key points of the reference image and the variation image;

[0048] S3: Use a random sampling consensus algorithm to purify the key point pairs obtained in step S2 to obtain multiple internal point sets and multiple homography matrices;

[0049] S4: determining the detection area according to the inliers in the set of all inliers, and mapping the key points in the reference image to the change image according to the obtained homography matrix, and then comparing these pairs of key points to obtain the changed key points;

[0050] S5: Calculate the maximum connected domain of the changed key points obtained in step S4, so as to obtain a change box;

[0051] S6: Setting a buffer to store the change box according to the context information of the change graph, and displaying the change box if the number of consecutive appearances of the change box in the buffer reaches a preset number threshold.

[0052] The implementation process of this method is as follows:

[0053] In the first step, set the maximum number of key points MaxKeyPointNum that you want to extract, use the key point detection method to detect key points in the image, select the first MaxKeyPointNum key points and calculate the descriptor. It includes the following steps:

[0054] Step S101, manually set parameters: the maximum number of key points to be extracted MaxKeyPointNum, and jump to step S1-2;

[0055] Step S102, using a key point detection method to detect key points in the image, the number of key points obtained is KeyPointNum, if KeyPointNum>MaxKeyPointNum, jump to step S103; otherwise jump to step S104;

[0056] Step S103, sort the key points according to their strengths, select the first MaxKeyPointNum key points, and jump to step S104.

[0057] Step S104, calculate the descriptor of the key point, and end step S1.

[0058] In the second step, set the search algorithm algorithm, the number of traversal checks, the distance threshold distanceRatio, build a fast nearest neighbor search matcher to match the key points in the reference image and the change image, and then filter out the wrong matches based on the distance threshold distanceRatio. It includes the following steps:

[0059] Step S201, manually set parameters: search algorithm algorithm, traversal times checks, distance threshold distanceRatio, and jump to step S202;

[0060] Step S202, construct a fast nearest neighbor search matcher according to the search algorithm algorithm and the traversal number checks, match the key points of the reference image and the change image, for each key point in the reference image, the matcher will find the two most matching key points in the change image, and jump to step S2-3;

[0061] Step S203, filter out false matches based on the distance threshold distanceRatio. distance <distanceRatio*n distance , then this is considered a correct match. distance is the Euclidean distance of the first match, n distance is the Euclidean distance of the second match, and step S2 ends;

[0062] In the third step, the RANSAC random sampling consensus algorithm is used to reduce the impact of outliers in the data. Since there may be multiple mapping relationships in a pair of images, when the proportion of outliers in the calculation of the homography matrix is ​​greater than the preset outlier ratio threshold threshold_outlier, we believe that there is still a mapping relationship in the outliers. At this time, it is necessary to continue to calculate the homography matrix based on the outliers. After the calculation is completed, we get multiple homography matrices. The multi-homography matrix calculation flow chart is shown in Figure 2 .

[0063] When calculating the homography matrix through RANSAC, "inliers" and "outliers" are distinguished, where "inliers" refer to points whose distance from the current fit is less than the threshold t, and "outliers" refer to abnormal data points. When the proportion of outliers in calculating the homography matrix is ​​greater than threshold_outlier, we believe that it is necessary to continue calculating the homography matrix. When calculating the next homography matrix, the previous inliers need to be removed from the previous matching points, that is, the points used when calculating the i+1th homography matrix are the initial matching points minus all inliers from the 1st to the i-th time. After the calculation is completed, we will get multiple inlier sets and multiple homography matrices, and each inlier set corresponds to a homography matrix.

[0064] In the fourth step, the detection area is determined based on all the inliers when calculating the homography matrix, and then the key points are re-divided at fixed intervals within the detection area. For the newly divided key points, it is necessary to determine which homography matrix should be used to map them from the reference image to the change image. This problem is essentially a key point classification problem, so we use the K nearest neighbor classification algorithm to determine the category based on the nearest few inliers, and then use the homography matrix corresponding to the category for mapping. Finally, the descriptors of the paired key points in the reference image and the change image are compared to obtain the key point pairs of the change.

[0065] In the fifth step, the maximum connected domain of the changed key points obtained in step 4 is calculated, and then the circumscribed rectangle of the maximum connected domain is used to represent the change box.

[0066] In the sixth step, we designed a buffer to utilize context information and propagate detection results to adjacent frames. The core idea of ​​the buffer is that only detection boxes that appear len times in a row will be displayed, and the displayed boxes will be removed from the buffer only if they do not appear len times in a row. The design diagram of the buffer is shown in Figure 3 .

[0067] For the change frame detected in the current frame, if the buffer is empty, directly store the change frame in the buffer and set the frequency to 1;

[0068] If the buffer is not empty, the change detection frames in the buffer are traversed to match the change frames of the current frame. If the match is successful, the frequency of the detection frame in the buffer is increased by one, and the attributes of the change frame are updated through average filtering. If frequency>len, the flag of the frame is set to true; if the match is not successful, the frequency of the current frame in the buffer is reduced by one. If frequency<0, the frame is removed from the buffer.

[0069] Considering the situation that a certain box is included by another box, for the boxes that are not matched in the current frame, first determine whether the box is included by the boxes in the buffer, and if so, mark the box as a match.

[0070] Finally, the change frames that are not matched in the current frame are added to the buffer as new change frames and all the change frames in the buffer with flag true are returned.

[0071] Verification example:

[0072] In order to verify the effectiveness of the method, this embodiment conducted change detection experiments on two image datasets, Hpatches and CDNet Dataset 2014. HPatches is a local descriptor evaluation index proposed in 2016, which contains 57 illumination folders and 59 viewpoint folders. Each folder contains a reference image and five images of the same scene taken under different illumination or different viewpoints, and also provides the homography matrix between the reference image and the change image. CDNet Dataset 2014 (CD2014) is improved on the basis of CDNet Dataset 2012. CDNet Dataset 2012 is a dataset containing 31 videos and approximately 90,000 manually annotated frames. CDNet Dataset 2014 adds five new categories. Each category contains 900 to 7,000 frames of images, and the resolution of the images ranges from 320*240 to 760*486. For each frame of the image, the CD2014 dataset provides a pixel-level GroundTruth label. Experiments on key point detection and multi-homography matrices are conducted on the Hpatches dataset, and experiments on the overall effect of change detection are conducted on the CD2014 dataset.

[0073] Using the above image dataset, the following steps are performed to evaluate homography estimation and change detection framework:

[0074] 1. Evaluation of homography estimation effect

[0075] S11: Set the maximum number of key points to be extracted MaxKeyPointNum to 1000, and select SIFT, Key.Net, and SuperPoint methods to perform key point detection on the Hpatches dataset.

[0076] S12: Set the search algorithm algorithm to FLANN_INDEX_KDTREE, the number of traversal checks to 10, the distance threshold distanceRatio to 0.8, and build a fast nearest neighbor search matcher to match the key points.

[0077] S13: Set the outlier ratio threshold threshold_outlier to 0.25, and then combine the RANSAC random sampling consensus algorithm to calculate the multi-homography matrix.

[0078] S14: Compare the key point repeatability and the proportion of key points that are successfully matched for the three methods under a single homography matrix and multiple homography matrices. The repeatability of a pair of key points a and b is defined as the distance between a and b after the homography matrix transformation is less than ε, and the proportion of key points that are successfully matched is defined as the proportion of inliers.

[0079] Figure 4 and Figure 5 They are the experimental results of homography estimation under illumination change and viewpoint change, respectively. Figure 4 It can be seen that in the scene with changing lighting, the homography estimation effects of Key.Net and SuperPoint are much better than SIFT when using one homography matrix, so multiple homography matrices have a greater improvement effect on SIFT, while Key.Net and SuperPoint have only a slight improvement. Figure 5 It can be found that the use of multiple homography matrices in the viewpoint change scene has a significant improvement on SIFT, SuperPoint and Key.Net, which is consistent with expectations. Because the viewpoint change may cause the image depth to change, breaking the constraints of traditional homography estimation, the effect of multiple homography matrices will be further improved under the viewpoint change. Using multiple homography matrices may cause a small decrease in the repeatability of key points, but it will greatly increase the proportion of successfully matched key points.

[0080] 2. Change Detection Effect Evaluation

[0081] S21 sets the maximum number of key points to be extracted (MaxKeyPointNum) to 1000, and selects SIFT, Key.Net, and SuperPoint to perform key point detection on the CDNet 2014 dataset.

[0082] S22 sets the search algorithm algorithm to FLANN_INDEX_KDTREE, the number of traversal checks to 10, the distance threshold distanceRatio to 0.8, and builds a fast nearest neighbor search matcher to match key points.

[0083] S23 sets the outlier ratio threshold threshold_outlier to 0.25, and then combines the RANSAC random sampling consensus algorithm to calculate the multi-homography matrix.

[0084] S24 determines the detection area according to all the inner points when calculating the homography matrix, and then re-divides the key points in the detection area according to the fixed interval. For the newly divided key points, the category is determined according to the nearest several inner points, and then the homography matrix corresponding to the category is used for mapping. Finally, the descriptors of the paired key points in the reference image and the change image are compared to obtain the changed key point pairs.

[0085] S25 calculates the maximum connected domain of the changed key points obtained in step S24, and then uses the circumscribed rectangle of the maximum connected domain to represent the change box.

[0086] S26 compares the change boxes obtained by the three methods with the Ground Truth provided by the CDNet 2014 dataset to measure the change detection effect. Assume that the set of change boxes predicted by the change detection framework is {a1,…,ak}, the set of change boxes obtained by GroundTruth is {b1,…,bn}, the total area of ​​the predicted box is Spred, the total area of ​​the GT box is Sgt, and the intersection area of ​​the predicted box and the GT box is Siou. Precision is defined here as how much of the predicted change area Spred is the real change area Siou, and Recall is defined as how much of the real change area Sgt is predicted.

[0087] Figure 6 is the result of the change detection experiment. Figure 6 It can be seen that the Precision, Recall and F1 indicators of SIFT, SuperPoint and Key.Net are not much different. On the contrary, SIFT has the best effect, Key.Net is the second, and SuperPoint is the worst. We analyzed the reasons for this result. This is the limitation of the CD2014 dataset, because the images in the dataset have neither viewpoint changes nor

[0088] When there are lighting changes, the traditional method of key point detection can achieve good results. The images themselves are aligned, and a homography matrix can be used to well describe the mapping relationship between the two images.

[0089] Although the quantitative experiments on the CD2014 dataset did not reflect the advantages of the framework of the present invention, in the qualitative experiments conducted on the test video, Key.Net, SuperPoint, multi-homography matrix and buffer all made the change detection effect better. Key.Net and SuperPoint made the key point coverage range larger and the number of successfully matched key points increased. The multi-homography matrix enabled the algorithm of the present invention to cope with the homography estimation in the scene where moving objects appeared in the image, and the buffer reduced the false detection and missed detection of change detection.

[0090] The present invention proposes a video stream change detection method based on image registration. Specifically, a close-up image change detection framework is designed using the steps of key point detection and matching, homography estimation, and difference detection. The key information in the image is fully utilized, and there is no need for paired data for training, which solves the problem of the dependence of the change detection task on paired data. On this basis, the present invention introduces a buffer method to utilize the contextual information of the video stream to avoid false detection and missed detection. Our experimental results on Hpatches show that SuperPoint and Key.Net have good robustness to illumination changes and viewpoint changes, respectively. Calculating multiple homography matrices in a pair of images can make the matching range of key points larger, which can solve the problems existing in traditional homography estimation. The qualitative experiments in the embodiments also prove that the buffer can reduce missed detection and false detection.

[0091] This embodiment also provides a video stream change detection system based on image registration, including a memory and a processor, wherein the memory stores a computer program, and the processor calls the computer program to execute the steps of the video stream change detection method based on image registration as described above.

[0092] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A video stream change detection method based on image registration, characterized in that: The following steps are involved: S1: Obtain a reference image and a variation image to be registered, and perform key point detection on the reference image and the variation image respectively; S2: Matching key points of the reference image and the variation image; S3: Use a random sampling consensus algorithm to purify the key point pairs obtained in step S2 to obtain multiple internal point sets and multiple homography matrices; S4: determining the detection area according to the inliers in the set of all inliers, and mapping the key points in the reference image to the change image according to the obtained homography matrix, and then comparing these pairs of key points to obtain the changed key points; S5: Calculate the maximum connected domain of the changed key points obtained in step S4, so as to obtain a change box; S6: setting a buffer to store the change box according to the context information of the change graph, and displaying the change box if the number of consecutive appearances of the change box in the buffer reaches a preset number threshold; The step S1 specifically includes: S101: Set the maximum number of key points to be extracted, and jump to step S102; S102: using a key point detection method to detect key points from the reference image and the variation image respectively, if the number of key points detected is greater than the maximum number of key points, jump to step S103, otherwise jump to step S104; S103: Sort the key points according to the strength of the detected key points, select the key points with the largest strength, and jump to step S104; S104: Calculate the descriptor of the key point; The step S2 specifically includes: S201: Select a search algorithm, set the number of traversal times and distance threshold, and jump to step S202; S202: According to the search algorithm and the number of traversals set in step S202, a fast nearest neighbor search matcher is constructed to match the key points of the reference image and the change image, and then the process goes to step S203; S203: Filtering out false matches of the matched key points according to the distance threshold.

2. The video stream change detection method based on image registration according to claim 1, characterized in that: In step S3, when calculating the homography matrix using the random sampling consistency algorithm, internal points and outliers are obtained. If the proportion of the outliers in the key points is greater than a preset outlier proportion threshold, the homography matrix is ​​continued to be calculated based on the outliers to obtain multiple homography matrices.

3. The video stream change detection method based on image registration according to claim 1 is characterized in that: The step S4 is specifically as follows: The detection area is determined according to the inliers in the set of all inliers, and the key points are re-divided at fixed intervals within the detection area. For the re-divided key points, the K nearest neighbor classification algorithm is used to determine the category of the current key point based on the nearest several inliers, so as to select the corresponding homography matrix for mapping; finally, the descriptors of the paired key points in the reference image and the changed image are compared to obtain the changed key point pairs.

4. The video stream change detection method based on image registration according to claim 1 is characterized in that: In step S5, a change frame is obtained according to the circumscribed rectangle of the largest connected domain.

5. The video stream change detection method based on image registration according to claim 1 is characterized in that: The step S6 specifically includes: setting a buffer to store the change box according to the context information of the change graph; if the change box in the buffer appears continuously for a number of times reaching a preset threshold of occurrences, the change box is displayed; if the displayed change box is not detected for a number of times reaching a preset threshold of non-appearances, the change box is removed from the buffer.

6. The video stream change detection method based on image registration according to claim 1, characterized in that: The step S6 specifically includes: For the change box detected in the current frame, if the buffer is empty, the change box is directly stored in the buffer and the number of occurrences is set to 1; if the buffer is not empty, the change boxes in the buffer are traversed to match the change boxes of the current frame. If the match is successful, the number of occurrences of the corresponding change box in the buffer is increased by 1. If the number of occurrences of the change box is greater than the preset number of occurrences threshold, the change box is displayed; the number of occurrences of the unmatched change box in the buffer is reduced by 1. If the number of occurrences of the change box in the buffer is less than 0, the corresponding change box is removed.

7. The video stream change detection method based on image registration according to claim 1 is characterized in that: The change graph is an image frame in a video stream.

8. A video stream change detection system based on image registration, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor calls the computer program to execute the steps of any one of the methods described in claims 1 to 7.

Images