Method and device for beat detection of deformable objects in a video stream

Abstract

The application provides a beat recognition method and device for a deformable object in a video stream, which does not perform target recognition on images in the video stream, but recognizes time frequencies of regular movements in the video stream, thereby efficiently extracting periodic changes in time frequencies of specific signals from the video stream. The beat recognition method for the deformable object in the video stream comprises the following steps: step 1, extracting matched feature points in adjacent two frames of the video stream by using a local matching and global screening method, and recursively calculating the feature points with consistency in each frame of pictures based on the matched feature points; step 2, performing motion statistical analysis on the feature points in multiple frames of pictures, and classifying the feature points into two categories, one category is background feature points with a significant similar movement rule, and the other category is deformable object feature points with a significant difference from the first category; and step 3, analyzing and recognizing periodic rules and quantitative beats from the movement rule of the deformable object feature points.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of video recognition, and particularly relates to a beat rhythm recognition method and device for a deformable object in a video stream. BACKGROUND

[0002] Most of the current algorithms related to video content recognition rely on machine learning to identify the main target of each frame. First, a large amount of training data set needs to be calculated, and second, the problem of poor neural network recognition accuracy caused by different target scales needs to be solved. Through feature engineering target recognition or deep neural network-based video stream analysis, the limited types of targets in the image are extracted first, and then motion law analysis is performed, which can achieve high precision. However, these neural network-based algorithms have large network parameter quantities and high requirements for device computing power. High computing power means high power consumption. For mobile devices with limited battery life, such as smartphones, tablets, etc., although this type of neural network-based algorithm can be used, it consumes power quickly, resulting in very limited battery life for mobile devices.

[0003] Specifically, for example, CN115601688A proposes a deep learning-based video main content detection method and system. The deep learning-based video main content detection method mainly acquires video data, adjusts the size of different targets in the video frame image to obtain a plurality of first initial images, forms an expanded data set, and obtains a first label image. The first target image is obtained by reasoning the expanded data set, the recognition accuracy of each category under different scale information is obtained by the edge difference between the first target image and the first label image, and the optimal reasoning scale of each category is obtained. The initial reasoning result is obtained by tile segmentation and recognition, the optimal reasoning scale corresponding to each connected domain is obtained, the scale is adjusted, and the optimal reasoning result is obtained. The main degree is obtained by calculating the multi-frame appearance degree, comprehensive area ratio and comprehensive central degree of the target area, and then the video main content is obtained. It can be seen that the deep learning algorithm has high requirements for the computing power of the device.

[0004] There is also a technology, for example, CN115690657A proposes an intelligent counting system based on video image monitoring, first, the video is stored, the data is input through the server input module, and the data of the input module is transmitted to the data storage, the data storage is subdivided into sub-storage A interval, sub-storage B interval and sub-storage C interval, the sub-storage B interval stores the video, the dynamic video object storage is stored through the video content, the sub-storage A interval of the data storage is used for inputting the captured object, the captured input object storage of the sub-storage A interval, through the captured input object storage, the high-suspected error reference object and the correct reference object are refined, and the high-suspected error reference object and the correct reference object are provided to the input object reference dynamic data capture comparison, and the target object is identified. In this way, the image data is saved more completely, but the data filtering efficiency is low, and the data processing efficiency is also low. SUMMARY

[0005] The present application is to solve the above problems, and aims to provide a beat recognition method and device for deformable objects in a video stream, which does not perform target recognition on images in the video stream, but recognizes the time frequency of regular motion in the video stream, thereby efficiently extracting the periodic change of the time frequency of a specific signal from the video stream.

[0006] In order to achieve the above purpose, the present application adopts the following scheme:

[0007] <Method>

[0008] As shown in Figure 1 The present application provides a beat recognition method for deformable objects in a video stream, comprising the following steps:

[0009] Step 1, using local matching and global screening method, extracting matched feature points in adjacent two frames of video stream, and based on this recursive calculation to obtain consistent feature points in each frame picture;

[0010] The feature point matching method is: for the adjacent two frames, the corner points of the first frame video are taken as the feature points, then according to the position of the feature points in the picture of the previous first frame video, a rectangular region with a certain height and width is taken as the center, and the matching corner points are found in this region as the feature points of the second frame video; according to the foregoing feature point matching method, all frames are processed until the last frame is iterated;

[0011] Step 2, motion statistical analysis is performed on the feature points in multiple frames, and the feature points are classified into two categories, one is the background feature point with obvious similar moving rule, and the other is the deformable object feature point which is obviously different from the first category;

[0012] Step 3, the periodicity and quantitative rhythm of the movable object feature point are identified from the motion law of the movable object feature point.

[0013] The beneficial effects of the scheme are:

[0014] Without target recognition of the image in the video stream, the motion law of the feature point in the image in the video stream is analyzed by statistical means, the feature point with regular motion is globally recognized without target recognition, and the time frequency of the regular motion in the video stream is recognized, thereby effectively solving the problem that the periodic change of the time frequency of the specific signal cannot be efficiently extracted from the video stream. The calculation amount is small, the power consumption is small, and the scheme can be better applied to mobile devices with limited endurance. At the same time, for ordinary devices, the energy consumption is low, and the scheme is more green and environmentally friendly.

[0015] The application scenarios of the scheme include but are not limited to: a, in the biotechnology field, the motion law of a biological body is analyzed through image signal statistics; b, in the medical field, the organ contraction and relaxation frequency, blood flow frequency, and electrocardiogram fluctuation frequency are analyzed through radiation, ultrasound, magnetic resonance, and other imaging methods; c, in the short video entertainment field, the periodicity of dance or motion is identified.

[0016] Preferably, the video stream movable object rhythm beat recognition method provided by the present application determines the size of the rectangular region in step 1 in the following manner:

[0017] Suppose the displacement speed of the movable object is v, and the displacement direction is θ, then the speed is decomposed as:

[0018]

[0019] Suppose the FPS of the video stream is f, and the resolution is expressed as: the number of pixel points in each row p x and the number of pixel points in each column p y ; the camera view angle is decomposed as:

[0020]

[0021] Suppose the video is shot at a distance d from the movable object, and the rectangular region defined in the feature point matching process has a length of w and a height of h, which are in pixels, so:

[0022]

[0023] The beneficial effects of the preferred technology are: the present application sets the size of the rectangular region by fully considering the influence of the movable object moving speed, the distance of the shooting object, the camera view angle, the resolution, and the FPS, and the rectangular region defined can effectively determine the position of the movable object.

[0024] Preferably, the beat recognition method of the deformable object in the video stream provided by the present application, in step 1, takes v as the limit displacement speed v of the deformable object max The size of the rectangular region is determined by the following formula:

[0025]

[0026] The preferred technology has the beneficial effect of simplifying the shooting process and further reducing the influence of measurement error on feature matching.

[0027] Preferably, the beat recognition method of the deformable object in the video stream provided by the present application, in step 1, calculates the displacement speed of the feature points after matching the feature points, and takes w i，j as the horizontal coordinate of the i-th feature point in the j-th frame, h i，j as the vertical coordinate of the i-th feature point in the j-th frame, to obtain the displacement speed v i，j and the displacement direction β i,j of the i-th feature point from the j-th frame to the j+1-th frame:

[0028]

[0029] The present application performs picture operation at the pixel level, effectively and reasonably cuts the video stream, is conducive to identifying and matching the deformable object, and further calculates the motion speed of the deformable object by using the above formula.

[0030] Preferably, the beat recognition method of the deformable object in the video stream provided by the present application adopts a square kernel template with the anchor point at the center, and the side length of the kernel template is a dynamic threshold (a typical value is the distance between the two closest feature points in all feature points). Then, a 3x3 light spot is established with each feature point as the center, the kernel template is used to perform an inflation operation on the edge of the light spot, and the number of connected domains is counted after each inflation. When the number of connected domains counted by adjacent two inflation operations is the same, the inflation operation is stopped. The light spot that has been connected is determined, and all feature points contained in each connected light spot are identified as similar repeated feature points. The feature point closest to the center of the light spot is taken as the effective feature point, and other feature points in the light spot are removed from the feature point set to be selected.

[0031] The preferred technology has the beneficial effect of effectively identifying information while greatly reducing repetitive work, saving memory, and further improving the speed of the algorithm.

[0032] Preferably, the beat identification method of the deformable object in the video stream provided by the present application, in step 1, when the rectangular region exceeds the actual picture boundary, the x coordinate of the starting point of the rectangle is changed to 0 when the rectangle exceeds the left edge of the image of each frame, the x coordinate of the starting point of the rectangle is changed to the length of each frame of the video picture minus the length of the rectangle when the rectangle exceeds the right edge, the y coordinate of the starting point of the rectangle is changed to 0 when the rectangle exceeds the upper edge, and the y coordinate of the starting point of the rectangle is changed to the width of each frame of the video picture minus the width of the rectangle when the rectangle exceeds the lower edge.

[0033] The beneficial effect of the preferred technique is that the part of the rectangular region of the feature point exceeding the actual picture boundary is removed, the repeated points are filtered, the number of correction times of the imaginary large number of edge rectangles is reduced, the feature points to be identified are finally determined, and the algorithm running time is further optimized.

[0034] Preferably, the beat identification method of the deformable object in the video stream provided by the present application, in step 2, the average speed of all feature points is calculated first The decentered speed of each feature point is obtained

[0035]

[0036]

[0037] Decentering speed fluctuation analysis: the distribution of the decentered speed of each point in the two-dimensional plane with time is solved to obtain a minimum circumscribed circle radius r i r of all feature points i Draw a statistical histogram; calculate the best segmentation threshold r' by dynamic threshold segmentation method, divide the histogram into two parts, the part on the left side of the histogram belongs to the feature points of the background, and the part on the right side belongs to the feature points of the deformable object. Through this method, the moving rule of the feature points can be effectively identified.

[0038] Preferably, the beat identification method of the deformable object in the video stream provided by the present application, in step 3, the relative speed of the second type of feature points belonging to the deformable object with time is curve fitted, and the time point of the extreme value of the curve is extracted; the beat information is the time median value experienced by the speed information of the deformable feature points from a peak point to a peak point in the opposite direction of the direction reaching the previous peak point, the beat information is determined by the change curve of the speed of the deformable feature points with time, the beat information is taken as the beat of the deformable object in the video stream.

[0039] <Device>

[0040] Further, the application also provides a beat recognition device for a deformable object in a video stream, which can automatically implement the above method, comprising:

[0041] The feature point matching extraction unit extracts matched feature points in two adjacent frames of the video stream by using a local matching and global screening method, and recursively calculates the consistent feature points in multiple frames of pictures; the feature point matching method is as follows: for two adjacent frames, the corner points of the first frame of video are taken as feature points, then a rectangular region with a certain height and width is cut out in the second frame of video according to the positions of the feature points of the first frame of video in the picture, and the matching corner points in the region are taken as the feature points of the second frame of video; all frames are processed according to the aforementioned feature point matching method until the last frame is iterated.

[0042] The feature point classification unit performs motion statistical analysis on the feature points in multiple frames of pictures, and classifies the feature points into two categories, one is the background feature points with a significant similar moving rule, and the other is the deformable object feature points with a significant difference from the first category.

[0043] The recognition unit analyzes and identifies the periodic rule and quantitative beat from the moving rule of the deformable object feature points.

[0044] The control unit is in communication connection with the feature point matching extraction unit, the feature point classification unit and the recognition unit, and controls the operation of them.

[0045] Preferably, the beat recognition device for a deformable object in a video stream provided by the application can further comprise an input display unit in communication connection with the control unit, for letting the user input operation instructions and performing corresponding display. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 A flowchart of the beat recognition method for a deformable object in a video stream related to the embodiments of the application;

[0047] Figure 2 A light spot expansion analysis process diagram related to the embodiments of the application;

[0048] Figure 3 A diagram of part of the feature points in a certain rectangular cutting region in the embodiments of the application (enlarged effect);

[0049] Figure 4 A diagram of the speed change of the deformable object and the background feature points related to the embodiments of the application;

[0050] Figure 5 A histogram division situation diagram related to the embodiments of the application, wherein the left part of r' in the diagram belongs to the background feature points, and the right part belongs to the feature points of the deformable object. DETAILED DESCRIPTION

[0051] The beat recognition method and device for deformable objects in a video stream according to the present application will be described in detail below with reference to the accompanying drawings.

[0052] <Embodiment One>

[0053] As shown in the figure, in this embodiment, the beat recognition method for deformable objects in a video stream provided by the present application is used for calculation, and the specific steps are as follows: Figure 1

[0054] Step 1: Use the local matching and global screening method to extract the matching feature points in the adjacent two frames of the video stream, and recursively calculate the consistent feature points in multiple frames of pictures.

[0055] In the case of having a common field of view, the relationship between the intersection parts of two synchronously collected (or non-synchronously collected for static scenes) pictures is a linear transformation. In this case, the feature points of the intersection parts of the two pictures can be directly extracted globally using the Fast-ORB method, and then registered through the RANSAC method. However, since the deformable objects in the adjacent two frames of the video stream will change randomly, the relationship between the common parts of the two images is not a linear change, so the Fast-ORB and RANSAC methods cannot be directly used to effectively register the key feature points in the two images. The present application uses the local matching and global screening method to distribute the registration of the feature points in the intersection parts of the adjacent two frames of the video stream, and the specific processing process is as follows:

[0056] 1) After determining the feature points based on the Fast-ORB method, the direction of the gray centroid in the local coordinate system is used to realize the rotation invariance of the descriptor, and the centroid and the mass center are defined by the following general mathematical expression of the moment:

[0057] m pq =∑ x,y x p y q I(x,y),

[0058] In the formula, I(x, y) is the gray density of the picture, so the normalized gray centroid position of the selected feature point is:

[0059]

[0060] A main direction for identifying the feature points is set according to the gray centroid:

[0061]

[0062] ​2) The neighborhood of the feature point is rotated to the principal direction, and the feature descriptor is constructed by Brief algorithm, thus the feature description vector of Fast-ORB is obtained.

[0063] 3) The corner points of the first frame of video are searched by Fast-ORB algorithm, then a rectangular region with a certain height and width is cut out according to the position of the corner points in the second frame of video, and the matching corner points are searched in the region. Similarly, after the matching feature points in the second frame are determined, the position of the feature points in the third frame is searched, and a rectangular region is determined according to the position and the matching corner points are searched, and the subsequent frames are processed in the above manner until the last frame is iterated.

[0064] 4) One of the key problems of recognizing deformable objects is how to determine the size of the rectangular region to be drawn. The size of the rectangular region determined by the application is affected by the moving speed of the deformable object, the distance of the shooting object, the camera angle, the resolution and the FPS. The method for determining the size of the rectangular region is as follows:

[0065] Suppose the displacement speed of the deformable object is v m / s, and the displacement direction is θ, that is, the angle between the displacement direction and the horizontal direction is θ, so the speed can be decomposed as:

[0066]

[0067] Since the FPS and the resolution of the video will affect the size of the rectangular region, the FPS of the video stream is set to f, which is fps (frame per second). The common FPS for video shooting is 24fps, 30fps or 60fps. In addition, the common resolutions are 720p, 1080p and 4k, which correspond to 1280*720, 1920*1080 and 4096*2160 pixel points in the same picture respectively. Set the former p x is the number of pixel points in each row, and the latter p y is the number of pixel points in each column.

[0068] Meanwhile, the camera angle also affects the feature point recognition of the deformable object. Since the CMOS and the video picture are rectangular, the camera angle can be decomposed as:

[0069]

[0070] Suppose the video is shot at a distance d from the deformable object, the unit is m, the rectangular region drawn in the feature point matching process has a length of w and a height of h, the unit is pixel, and the size of the rectangular region is calculated by the following formula:

[0071]

[0072] The defined rectangular region effectively determines the position of the deformable object.

[0073] In order to simplify the shooting process and reduce the influence of measurement error on feature matching, v in the above formula can be taken as the limit displacement speed v of the deformable object max , so that:

[0074]

[0075] The Fast-ORB algorithm is used to screen and pair the feature points in the rectangular region with a length of w and a width of h, so as to determine the feature point pair information of each adjacent two frames.

[0076] After the feature points are matched, the displacement speed of the feature points can be calculated, and w i，j is the horizontal coordinate of the i-th feature point in the j-th frame, and h i，j is the vertical coordinate of the i-th feature point in the j-th frame. The displacement speed v i，j and the displacement direction β i，j of the i-th feature point from the j-th frame to the j+1-th frame can be obtained, that is:

[0077]

[0078] In the recognition algorithm, the present application performs picture operation at the pixel level, effectively and reasonably cuts the video stream, so that the deformable object can be recognized and matched, and further, in order to recognize the beat, the change rule of the motion speed of the deformable object should be calculated.

[0079] After the center of the rectangular region to be processed and the size of the rectangular region are determined, the pictures in the region of the two frames are extracted using a mask, and the target extraction of the matching region is completed.

[0080] Since the picture is cut in the corresponding feature region when a feature point is recognized, there are multiple feature points in the feature region. If all the feature points in the region are recognized and the deformable object is recognized, there will be a large number of repeated feature point pairs. When there are a large number of frames in the video, a large number of repeated feature point data will be recognized and stored, so the present application proposes to set a feature point pair threshold, such as Figure 2As shown, a 3X3 star-shaped light spot is established around each feature point, then the light spot is dilated and connected domain analysis is performed, when the number of dilations reaches a threshold, the light spot that has been connected is determined, and the feature point represented by the light spot is identified as a repeated feature point, the feature point at the center position is taken as an effective feature point, and other feature points in the light spot are removed from the set of feature points to be selected, thereby greatly reducing the repeated work of the algorithm, saving storage space, and improving the implementation speed of the algorithm.

[0081] In particular, when the rectangular region of the feature point exceeds the actual picture boundary, a part of the rectangular region does not have actual meaning, so when the rectangular region exceeds the boundary, the x coordinate of the starting point of the rectangular region is changed to 0 when the rectangular region exceeds the left edge of each frame image, the x coordinate of the starting point of the rectangular region is changed to the length of each frame image of the video minus the length of the rectangular region when the rectangular region exceeds the right edge, the y coordinate of the starting point of the rectangular region is changed to 0 when the rectangular region exceeds the upper edge, the y coordinate of the starting point of the rectangular region is changed to the width of each frame image of the video minus the width of the rectangular region when the rectangular region exceeds the lower edge, other feature points in this region are optimized according to the light spot dilation algorithm described above, which can filter many repeated points and reduce the number of corrections of the imaginary large number of edge rectangles, and finally determine the feature points to be recognized, thereby optimizing the algorithm running time.

[0082] Step 2: As shown in Figures 3 to 5 The feature points in the intersection part of the multiple frames of images in the video stream are matched, and the feature points are classified into two categories, one category is background feature points with a significant similar movement rule, and the other category is deformable object feature points with a significant difference from the first category.

[0083] After matching the feature points in the intersection part of the multiple frames of images in the video stream, the moving speed of the feature points in the image sequence can be calculated. According to the speed rule, these feature points can be roughly divided into two categories, one category is background feature points, these points have a significant similar speed rule; the second category is deformable object feature points, the speed of this category of feature points has a significant difference from the speed rule of the first category, and the speed distribution is not concentrated. The present application utilizes the speed rule to classify these feature points into two categories, and the specific processing process is as follows:

[0084] 1) Speed decentralization. By first calculating the average speed of all feature points The speed of each feature point after decentralization is obtained

[0085]

[0086]

[0087]

[0088]

[0089] In the formula, n is the total number of feature points;

[0090] 2) Decentralized speed fluctuation analysis. The distribution of the decentralized speed of each point over time in a two-dimensional plane is solved to obtain a minimum circumscribed circle radius r i r of all feature points i Draw a statistical histogram. The best segmentation threshold r' is calculated using a dynamic threshold segmentation method similar to image processing, and the histogram is divided into two parts, as shown in Figure 5 Where the left half belongs to the background feature points, and the right half belongs to the deformable object feature points.

[0091] Step 3: Analyze the periodicity and quantitative rhythm of the deformable object feature points from the motion law of the deformable object feature points.

[0092] The third step of the present application is to identify two types of feature points in the multi-frame image of the video stream, and then use the relative speed method to identify the periodicity of the second type of feature points belonging to the deformable object. The specific processing process is as follows:

[0093] Curve fitting is performed on the relative speed of the second type of feature points belonging to the deformable object over time, and the time point at which the extreme value of the curve occurs is extracted.

[0094] The beat information is represented as the time median experienced by the speed information of the deformable feature points from a peak point to a peak point in the opposite direction of the direction reaching the previous peak point. Therefore, the beat information can be determined through the speed-time curve of the deformable feature points. Take the beat analysis of all deformable object feature points, and take the average of the beat information as the final rhythm beat of the deformable object in the video stream.

[0095] <Embodiment Two>

[0096] In this embodiment two, a device for automatically identifying the rhythm beat of the deformable object in the video stream according to the method of the present application is also provided. The device includes a feature point matching extraction unit, a feature point classification unit, an identification unit, an input display unit, and a control unit.

[0097] The feature point matching extraction unit performs the content described in step 1 above, extracts the matching feature points in the adjacent two frames of the video stream by using the local matching and global screening method, and recursively calculates the consistent feature points in multiple frames of pictures; the feature point matching mode is as follows: for the adjacent two frames, the corner points of the first frame video are taken as the feature points, then a rectangular region with a certain height and width is cut around the position of the feature points in the first frame video in the second frame video, and the matching corner points in the region are taken as the feature points of the second frame video; all frames are processed according to the foregoing feature point matching mode until the last frame is iterated.

[0098] The feature point classification unit performs the content described in step 2 above, performs motion statistical analysis on the feature points in multiple frames of pictures, and classifies the feature points into two categories, one is the background feature point with significant similar movement rule, and the other is the deformable object feature point with significant difference from the first category.

[0099] The recognition unit performs the content described in step 3 above, analyzes and identifies the periodic rule and quantitative rhythm from the movement rule of the deformable object feature points.

[0100] The input display unit is used for inputting operation instructions and performing corresponding display. Specifically, the input display unit can display the input, output and intermediate processing data of the feature point matching extraction unit, the feature point classification unit and the recognition unit according to the operation instructions.

[0101] The control unit is in communication connection with the feature point matching extraction unit, the feature point classification unit, the recognition unit and the input display unit, and controls the operation of them.

[0102] The above embodiments are only examples of the technical solutions of the present application. The rhythm beat recognition method and device of the deformable object in the video stream involved in the present application are not limited to the content described in the above embodiments, but are subject to the scope defined in the claims. Any modification, supplement or equivalent replacement made by the skilled in the art on the basis of the embodiments is within the scope claimed by the claims of the present application.

Claims

1. A method for beat detection of a deformable object in a video stream, characterized in that, The method comprises the following steps: Step 1, using local matching and global screening method, extracting the matching feature points in the adjacent two frames of the video stream, and recursively calculating the consistent feature points in each frame based on this; The feature point matching method is as follows: for the adjacent two frames, the corner points of the first frame video are taken as the feature points, then a rectangular region with a certain height and width is cut out according to the position of the feature points in the first frame video in the picture, and the matching corner points in the region are taken as the feature points of the second frame video; all frames are processed according to the foregoing feature point matching method until the last frame is iterated; Step 2, motion statistical analysis is performed on the feature points in multiple frames, and the feature points are classified into two categories: one is the background feature points with significant similar movement law, and the other is the deformable object feature points which are obviously different from the first category; Step 3, the periodic law and quantitative rhythm of the deformable object feature points are analyzed and identified from the movement law of the deformable object feature points.

2. The rhythm beat recognition method of the deformable object in the video stream according to claim 1, characterized in that: wherein In step 1, the size of the rectangular region is determined as follows: Let the displacement speed of the deformable object be v, and the displacement direction be θ, then the speed is decomposed as: Let the FPS of the video stream be f, and let the resolution be represented as: the number of pixels per row p x and the number of pixels per column p y ; to the camera view breaks down to: Let the video be shot at a distance d from the deformable object, and the rectangular region defined in the feature point matching process has a length of w and a height of h, which are in pixels, so:

3. The rhythm beat recognition method of the deformable object in the video stream according to claim 2, characterized in that: wherein In step 1, v is taken as the limit displacement velocity v of the deformable object max The size of the rectangular region is determined using the following formula:

4. The rhythm beat recognition method of the deformable object in the video stream according to claim 2, characterized in that: wherein In step 1, the displacement velocity of the feature points is calculated after matching the feature points, let w i,j be the horizontal coordinate of the i-th feature point in the j-th frame, h i,j be the vertical coordinate of the i-th feature point in the j-th frame, the displacement velocity v i,j and the displacement direction β i,j of the i-th feature point from the j-th frame to the j+1-th frame are obtained:

5. The rhythm beat recognition method of the deformable object in the video stream according to claim 1, characterized in that: wherein, In the process of identifying feature points in step 1, a square kernel template with an anchor point at the center is used, then a 3x3 light spot is established around each feature point, the kernel template is selected, and the edge of the light spot is dilated; when the number of connected domains obtained by two adjacent dilations is the same, the dilation operation is stopped; the connected light spots are determined, and all feature points contained in each connected light spot are identified as similar repeated feature points; the feature point closest to the center of the light spot is taken as the effective feature point, and the other feature points in the light spot are removed from the feature point set to be selected.

6. The rhythm beat recognition method of the deformable object in the video stream according to claim 1, characterized in that: wherein In step 1, when the rectangular region exceeds the actual picture boundary, the x coordinate of the starting point of the rectangular region is changed to 0 when the rectangular region exceeds the left edge of each frame image, the x coordinate of the starting point of the rectangular region is changed to the length of each frame image of the video minus the length of the rectangular region when the rectangular region exceeds the right edge, the y coordinate of the starting point of the rectangular region is changed to 0 when the rectangular region exceeds the upper edge, and the y coordinate of the starting point of the rectangular region is changed to the width of each frame image of the video minus the width of the rectangular region when the rectangular region exceeds the lower edge.

7. The beat-detection method of a deformable object in a video stream of claim 1, characterized in that: wherein, In step 2, the average velocity of all feature points is calculated first The decentered velocity of each feature point is obtained Decentralized speed fluctuation analysis: the distribution of the decentralized speed of each point in the two-dimensional plane as a function of time is solved to obtain a minimum circumscribed circle radius r i r of all feature points i Draw a statistical histogram; The best segmentation threshold r' is calculated by dynamic threshold segmentation method, and the histogram is divided into two parts. The feature points on the left side of r' belong to the background, and the feature points on the right side belong to the deformable object.

8. The beat identification method of the deformable object in the video stream according to claim 1, characterized in that: wherein In step 3, the relative speed of the second type of feature points belonging to the deformable object is curve-fitted with time, and the time point of the extreme value of the curve is extracted; the beat information is the time median value experienced by the speed information of the deformable feature points from a peak point to a peak point in the opposite direction of the direction to the previous peak point; the beat information is determined by the speed-time curve of the deformable feature points, and the beat of the deformable object in the video stream is obtained by averaging the beat information of all deformable object feature points.

9. Apparatus for beat recognition of a deformable object in a video stream, characterized in that, Comprise: The feature point matching extraction unit extracts the matching feature points in the adjacent two frames of the video stream by local matching and global screening method, and recursively calculates the consistent feature points in multiple frames of pictures; The feature point matching method is as follows: for the adjacent two frames, the corner points of the first frame video are taken as the feature points, then a rectangular region with a certain height and width is cut out according to the position of the feature points in the first frame video in the second frame video, and the matching corner points in the region are taken as the feature points of the second frame video; all frames are processed according to the above feature point matching method until the last frame is iterated; The feature point classification unit performs motion statistical analysis on the feature points in multiple frames of pictures, and classifies these feature points into two categories: one is the background feature point with obvious similar movement rule, and the other is the deformable object feature point with obvious difference from the first category; The identification unit analyzes and identifies the periodicity and quantitative beat from the movement rule of the deformable object feature points; The control unit is in communication with the feature point matching extraction unit, the feature point classification unit and the identification unit, and controls the operation of them.

10. The apparatus according to claim 9, wherein Further comprise: The input display unit is in communication with the control unit, and is used for inputting operation instructions and corresponding display.

Images