Video detection method and device, electronic device, and storage medium

Abstract

The present disclosure relates to a video detection method and device, electronic equipment and storage medium. The method comprises: obtaining displacement information between adjacent first frame image and second frame image in a video image; wherein the displacement information comprises displacement generated by non-linear motion of a device collecting the first frame image and the second frame image; determining the anti-shake effect of the video image according to the displacement information. Through the method, the accuracy of video anti-shake effect detection can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer vision technology, and in particular to a video detection method and apparatus, electronic device and storage medium. Background Technology

[0002] Existing display devices (such as mobile phones or tablets) typically perform image stabilization when displaying video.

[0003] However, when evaluating the effectiveness of video stabilization, it is usually based on the subjective evaluation of testers, which is labor-intensive and results in large differences due to inconsistent evaluation standards. Summary of the Invention

[0004] This disclosure provides a video detection method and apparatus, an electronic device, and a storage medium.

[0005] According to a first aspect of the present disclosure, a video detection method is provided, comprising:

[0006] The displacement information between adjacent first and second frames in a video image is obtained; wherein, the displacement information includes the displacement caused by the non-linear motion of the device acquiring the first and second frames;

[0007] The stabilization effect of the video image is determined based on the displacement information.

[0008] In some embodiments, obtaining the displacement information between adjacent first and second frame images in a video image includes:

[0009] A first vector is determined based on the preset coordinate origin and the first displacement information included in the preset transformation matrix; wherein, the first vector is characterized as the vector of the first displacement information relative to the preset coordinate origin, and the first displacement information is the translation information of the pixels in the first frame image and the second frame image;

[0010] The first vector is rotated by different preset radii to obtain a second vector included in a circle of each preset radius; wherein the directions of the second vectors in the circles of the preset radii are different.

[0011] For circles with preset radii, the average displacement of each second vector on the first frame image and the second frame image is determined based on each second vector, the preset transformation matrix, and the weight information corresponding to the preset radius; wherein, the weight information corresponding to the preset radius is used to characterize the degree of attention to the image content in the first frame image;

[0012] The displacement information is determined based on the average displacement of the second vector in circles with different preset radii on the first frame image and the second frame image.

[0013] In some embodiments, determining the average displacement of each second vector in the first frame image and the second frame image based on the weight information corresponding to each second vector, the preset transformation matrix, and the preset radius includes:

[0014] Multiply each of the second vectors by the preset transformation matrix to obtain a first value that maps each of the second vectors onto the second frame image;

[0015] Obtain the difference between the sum of each of the first values ​​and the sum of each of the second vectors;

[0016] Obtain the quotient of the difference and the number of the second vectors;

[0017] The average displacement is obtained by multiplying the quotient by the weight information corresponding to the preset radius.

[0018] In some embodiments, determining the displacement information based on the average displacement of the second vector in circles with different preset radii across the first frame image and the second frame image includes:

[0019] The displacement information is determined by summing the average displacement of the second vector in circles with different preset radii on the first frame image and the second frame image.

[0020] In some embodiments, determining the first vector based on the preset coordinate origin and the first displacement information included in the preset transformation matrix includes:

[0021] If the preset coordinate origin is a point outside the center of the first frame image, the first vector is determined based on the positional relationship between the preset coordinate origin and the center of the first frame image and the first displacement information; wherein, the first vector is the vector of the first displacement information relative to the center of the first frame image.

[0022] In some embodiments, the method further includes:

[0023] Corner detection is performed on the first frame image and the second frame image to obtain the first corner point corresponding to the first frame image and the second corner point corresponding to the second frame image;

[0024] Corner point matching is performed on the first corner point and the second corner point, and the preset transformation matrix is ​​calculated based on the matched corner points.

[0025] In some embodiments, the step of performing corner point matching on the first corner point and the second corner point, and calculating the preset transformation matrix based on the matched corner points, includes:

[0026] Corner point matching is performed on the first corner point and the second corner point based on optical flow method to obtain preliminary matching corner points;

[0027] The preliminary matching corner points are filtered based on the Random Sampling Consensus (RANSAC) method to obtain the target matching corner points;

[0028] The preset transformation matrix is ​​calculated based on the target matching corner points.

[0029] In some embodiments, the step of filtering the initial matching corner points based on the Random Sampling Consensus (RANSAC) method to obtain target matching corner points includes:

[0030] The target matching corner is obtained by performing a preset K rounds of filtering on the initial matching corner based on the RANSAC method; wherein, K is a positive integer greater than 1, and the matching corner obtained in the previous filtering based on the RANSAC method is the corner to be filtered in the next execution of the RANSAC method.

[0031] In some embodiments, determining the image stabilization effect of the video image based on the displacement information includes:

[0032] The calculated value is determined based on the displacement information;

[0033] The image stabilization effect of the video image is determined based on the calculated value; wherein the calculated value is inversely proportional to the image stabilization effect, and the calculated value includes at least one of the following:

[0034] The mean value of displacement information corresponding to different adjacent images;

[0035] Standard deviation of displacement information corresponding to different adjacent images;

[0036] The median value of displacement information corresponding to different adjacent images.

[0037] In some embodiments, the method further includes:

[0038] The device receiving the video image sent by the device that acquired the first frame image and the second frame image; wherein the video image includes the first frame image and the second frame image;

[0039] After determining the stabilization effect of the video image based on the displacement information, the stabilization effect is sent to the device.

[0040] According to a second aspect of the present disclosure, a video detection apparatus is provided, comprising:

[0041] The acquisition module is configured to acquire displacement information between adjacent first and second frame images in a video image; wherein, the displacement information includes displacement caused by the non-linear motion of the device acquiring the first and second frame images;

[0042] The determination module is configured to determine the image stabilization effect of the video image based on the displacement information.

[0043] In some embodiments, the acquisition module is further configured to determine a first vector based on a preset coordinate origin and first displacement information included in a preset transformation matrix; wherein the first vector represents the vector of the first displacement information relative to the preset coordinate origin, and the first displacement information is the translation information of pixels in the first frame image and the second frame image; rotate the first vector by different preset radii to obtain second vectors included in circles of each preset radius; wherein the directions of the second vectors in the circles of each preset radius are different; for each circle of preset radius, determine the average displacement of each second vector in the first frame image and the second frame image based on each second vector, the preset transformation matrix, and the weight information corresponding to the preset radius; wherein the weight information corresponding to the preset radius is used to represent the degree of attention to the image content in the first frame image; determine the displacement information based on the average displacement of the second vectors in the circles of different preset radii in the first frame image and the second frame image.

[0044] In some embodiments, the acquisition module is further configured to multiply each of the second vectors by the preset transformation matrix to obtain a first value of each of the second vectors mapped onto the second frame image; obtain the difference between the sum of the first values ​​and the sum of the second vectors; obtain the quotient of the difference and the number of the second vectors; and multiply the quotient by the weight information corresponding to the preset radius as the average displacement.

[0045] In some embodiments, the acquisition module is further configured to determine the displacement information as the sum of the average displacements of the second vectors in circles with different preset radii on the first frame image and the second frame image.

[0046] In some embodiments, the acquisition module is further configured to determine the first vector based on the positional relationship between the preset coordinate origin and the center of the first frame image and the first displacement information if the preset coordinate origin is a point outside the center of the first frame image; wherein the first vector is the vector of the first displacement information relative to the center of the first frame image.

[0047] In some embodiments, the apparatus further includes:

[0048] The detection module is configured to perform corner detection on the first frame image and the second frame image to obtain the first corner point corresponding to the first frame image and the second corner point corresponding to the second frame image;

[0049] The matching module is configured to perform corner point matching on the first corner point and the second corner point, and calculate the preset transformation matrix based on the matched corner points.

[0050] In some embodiments, the matching module is further configured to perform corner matching on the first corner point and the second corner point based on optical flow to obtain preliminary matching corner points; filter the preliminary matching corner points based on the Random Sampling Consensus (RANSAC) method to obtain target matching corner points; and calculate the preset transformation matrix based on the target matching corner points.

[0051] In some embodiments, the matching module is further configured to perform a preset K-round filtering on the initial matching corner points based on the RANSAC method to obtain the target matching corner points; wherein, K is a positive integer greater than 1, and the matching corner points obtained in the previous filtering based on the RANSAC method are the corner points to be filtered in the next execution of the RANSAC method.

[0052] In some embodiments, the determining module is further configured to determine a calculated value based on the displacement information; and to determine the image stabilization effect of the video image based on the calculated value; wherein the calculated value is inversely proportional to the image stabilization effect, and the calculated value includes at least one of the following:

[0053] The mean value of displacement information corresponding to different adjacent images;

[0054] Standard deviation of displacement information corresponding to different adjacent images;

[0055] The median value of displacement information corresponding to different adjacent images.

[0056] In some embodiments, the apparatus further includes:

[0057] The receiving module is configured to receive the video image sent by the device that acquired the first frame image and the second frame image; wherein the video image includes the first frame image and the second frame image;

[0058] The sending module is configured to send the image stabilization effect to the device after determining the stabilization effect of the video image based on the displacement information.

[0059] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0060] processor;

[0061] Memory used to store processor-executable instructions;

[0062] The processor is configured to perform the video detection method as described in the first aspect above.

[0063] According to a fourth aspect of the present disclosure, a storage medium is provided, comprising:

[0064] When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the video detection method as described in the first aspect above.

[0065] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0066] In the embodiments of this disclosure, the video stabilization effect is determined by automatically analyzing adjacent frame images using electronic devices, without relying on subjective testing. This reduces the influence of subjective factors introduced by subjective testing. Furthermore, this disclosure calculates the displacement information between adjacent frame images caused by the non-linear motion of the device acquiring the video images, and then determines the video stabilization effect based on this displacement information. This balances the impact of non-linear motion (rotational motion) on video jitter, thus improving the accuracy of video stabilization effect detection.

[0067] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0068] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0069] Figure 1 This is a video detection method flow shown in an embodiment of the present disclosure. Figure 1 .

[0070] Figure 2 This is a video detection method flow shown in an embodiment of the present disclosure. Figure 2 .

[0071] Figure 3 This is an example diagram illustrating the circle-drawing method according to an embodiment of this disclosure.

[0072] Figure 4 This is a flowchart of a method for determining a preset transformation matrix in an embodiment of this disclosure.

[0073] Figure 5 This is a video detection method flow shown in an embodiment of the present disclosure. Figure 3 .

[0074] Figure 6 This is a schematic diagram illustrating the principle of video detection according to an embodiment of the present disclosure.

[0075] Figure 7 This is a video detection method flow shown in an embodiment of the present disclosure. Figure 4 .

[0076] Figure 8 This is an example diagram of a video detection application scenario in an embodiment of this disclosure.

[0077] Figure 9 This is a diagram illustrating a video detection device according to an exemplary embodiment.

[0078] Figure 10 This is a block diagram illustrating a terminal according to an exemplary embodiment.

[0079] Figure 11 This is a block diagram illustrating a server according to an exemplary embodiment. Detailed Implementation

[0080] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0081] Figure 1 This is a video detection method flow shown in an embodiment of the present disclosure. Figure 1 ,like Figure 1 As shown, the video detection method applied to electronic devices includes the following steps:

[0082] S11. Obtain displacement information between adjacent first and second frame images in the video image; wherein, the displacement information includes displacement caused by the non-linear motion of the device acquiring the first and second frame images;

[0083] S12. Determine the stabilization effect of the video image based on the displacement information.

[0084] In embodiments of this disclosure, the electronic device may be a terminal, including mobile phones, tablet computers, or personal computers (PCs), etc., and the terminal may be a terminal for capturing or displaying video images.

[0085] Considering that video may involve some rotational motion during the shooting process, such as non-linear motion of the device capturing video images due to the movement of the user's hand, this disclosure obtains displacement information between adjacent first and second frame images in the video image in step S11. This displacement information includes the displacement caused by the non-linear motion of the device.

[0086] For example, when the device capturing video images does not move, the same target is in the same position in adjacent frames. If there is linear motion, the same target is translated in adjacent images, such as translation in any direction (up, down, left, right), that is, the pixels between adjacent images are translated. If there is non-linear motion, the same target is displaced in multiple directions in adjacent frames, that is, the pixels between adjacent images are displaced in multiple directions.

[0087] In this disclosure, after acquiring the displacement information between the first frame and the second frame, the stabilization effect of the video image is determined in step S12 based on the displacement information. It is understood that since the displacement information includes the displacement caused by the non-linear motion of the device acquiring the first and second frames, thus taking into account the jitter caused by, for example, rotational motion, the accuracy of video stabilization effect detection can be improved.

[0088] It should be noted that the stabilization effect of a video image can be a rating of the stabilization effect. For example, the less the video image shakes, the higher the rating and the better the stabilization effect.

[0089] In related technologies, the effectiveness of video stabilization or areas for improvement are typically evaluated through subjective assessments by testers. The problem with subjective testing is that different testers may have different evaluation criteria, leading to different test results for the same video from different individuals, and it also consumes significant human resources.

[0090] It is understood that this disclosure does not rely on subjective testing. Instead, it uses automated analysis and processing of adjacent frame images by electronic devices to determine the video stabilization effect, thus reducing the influence of subjective factors introduced by subjective testing. Furthermore, calculating displacement information between adjacent frame images, including that caused by non-linear device motion, before determining the video stabilization effect can balance the impact of rotational motion on video jitter, thereby improving the accuracy of video stabilization effect detection.

[0091] It should be noted that, in the disclosed embodiments, the adjacent first and second frames in a video image can be any two adjacent frames in the video image. The electronic device of this disclosure can perform the above processing on all adjacent frames in a video image to determine the video stabilization effect. Furthermore, the adjacent first and second frames can also be adjacent frames within a specified time period, for example, adjacent frames located in the middle of a time period in a video image. The electronic device determines the video stabilization effect based on these specified two frames.

[0092] Figure 2 This is a flowchart of a video detection method in an embodiment of the present disclosure. Figure 2 ,like Figure 2 As shown, in Figure 1 Based on the method shown, step S11 includes the following steps:

[0093] S11a. Determine a first vector based on the first displacement information included in the preset coordinate origin and the preset transformation matrix; wherein, the first vector is characterized as the vector of the first displacement information relative to the preset coordinate origin, and the first displacement information is the translation information of the pixel points in the first frame image and the second frame image;

[0094] S11b. Rotate the first vector by different preset radii to obtain a second vector included in a circle of each preset radius; wherein the directions of the second vectors in the circles of the preset radii are different.

[0095] S11c. For circles with preset radii, determine the average displacement of each second vector on the first frame image and the second frame image based on each second vector, the preset transformation matrix, and the weight information corresponding to the preset radius; wherein, the weight information corresponding to the preset radius is used to characterize the degree of attention to the image content in the first frame image.

[0096] S11d. Determine the displacement information based on the average displacement of the second vector in circles with different preset radii on the first frame image and the second frame image.

[0097] It should be noted that, in this embodiment, the preset transformation matrix can be a transformation matrix obtained from one or more sets of adjacent frame images in the video image. This transformation matrix can be an affine transformation matrix, which may include information such as image translation, rotation, and scaling. The following formula (1) is the transformation matrix A mentioned in this disclosure:

[0098]

[0099] Where s represents scaling, θ represents rotation angle, and (t x ,t y) indicates translation information.

[0100] In the above formula (1), (t) x ,t y The translation information represented by ) is the first displacement information mentioned in this disclosure. The first displacement information represents the positional translation of the same image content in adjacent first frame images and second frame images, that is, the translation of pixel points.

[0101] In this embodiment, when calculating the displacement information generated by non-linear motion based on the first displacement information, a first vector is first determined based on the first displacement information. This first vector is the vector of the first displacement information relative to a preset coordinate origin. The preset coordinate origin is the coordinate origin used when calculating the preset transformation matrix.

[0102] In one embodiment, if the preset origin for calculating the transformation matrix is ​​the center of the image (both the first and second frame images use the image center as the origin), then the first vector is the image center and the first displacement information (t). x ,t y A vector composed of points. In embodiments of this disclosure, the first vector is represented by vector v.

[0103] In another embodiment, determining the first vector based on the preset coordinate origin and the first displacement information included in the preset transformation matrix includes:

[0104] If the preset coordinate origin is a point outside the center of the first frame image, the first vector is determined based on the positional relationship between the preset coordinate origin and the center of the first frame image and the first displacement information; wherein, the first vector is the vector of the first displacement information relative to the center of the first frame image.

[0105] For example, if the preset coordinate origin is the top left corner of the image (both the first and second frame images use the top left corner of the image as the origin), then the first vector is determined according to the positional relationship between the preset coordinate origin and the center of the image as shown in the following formula (2):

[0106] v = v e +[w / 2,h / 2] (2)

[0107] Where w is the width of the first frame image, h is the height of the first frame image, and the dimensions of each frame image in the video are consistent; v e For initial displacement information (t) x ,t y The vector pointed to by ) is the first vector v, which is t. x Translate by w / 2, t along the horizontal axis y The vector formed by combining the point after translation h / 2 along the vertical axis with the center of the image.

[0108] After determining the first vector, this disclosure allows the first vector to be rotated with a preset radius and a preset direction. It should be noted that in this embodiment, rotating the first vector with different preset radii yields second vectors within circles of each preset radius, where the directions of the second vectors within the circles of the preset radii are different. The following formula (3) represents the second vector v' corresponding to a counterclockwise rotation of the first vector with a radius of 1 / 2 (e.g., the length of the first vector itself) at a preset angle φ:

[0109]

[0110] In the embodiments of this disclosure, different preset radii and rotation angles can be set according to requirements, such as in the above formula (3). v is rotated counterclockwise by an angle φ to obtain v', and v' is then rotated counterclockwise by φ according to the above formula (3) to obtain v''. This iterative process can yield a circle. To satisfy the symmetry of the circle (the symmetry of the sampling points on the circle), the angle φ needs to satisfy the following conditions:

[0111] 360° mod φ = 0 (4)

[0112] It should be noted that, in the embodiments disclosed herein, as many preset radii and rotation angles as possible can be set to cover the field of view of the image as much as possible, thereby improving the accuracy of displacement information estimation caused by non-linear motion.

[0113] For circles with preset radii, after determining the second vectors corresponding to each rotation of the first vector around the preset radius, the average displacement of each second vector can be determined based on the weight information corresponding to the preset radius, the preset transformation matrix, and the second vector. Then, based on the average displacement of the second vectors within each circle with a preset radius, the average displacement of the entire image is obtained, thus yielding the aforementioned displacement information. The weight information corresponding to the preset radius represents the degree of attention paid to the image content, which can be, for example, content located in the center of the image or content located at the edge of the image.

[0114] For example, if more attention is paid to the edges of the image, the weight corresponding to the circle with a radius of 1 / 2 can be set to be larger, thereby obtaining displacement information that is more biased towards the edges; if different image contents in the image are not distinguished, the weights corresponding to circles of different radii can be set to be the same, and the resulting displacement information can characterize the displacement of the image center. In this embodiment of the disclosure, the sum of the weight information corresponding to circles with different preset radii is 1.

[0115] The method for obtaining displacement information disclosed above can be called the circle drawing method. Figure 3 This is an example diagram illustrating the circle-drawing method according to an embodiment of this disclosure, such as... Figure 3As shown, each second vector can be determined by combining the transformation matrix, and then the displacement can be determined by comprehensively considering the weight information corresponding to different preset radii.

[0116] Understandably, in this embodiment, the first vector is simulated by drawing circles to obtain second vectors included in circles with different preset radii. Then, the average displacement of each second vector is determined for each circle with a preset radius. The displacement information is then comprehensively determined based on the average displacement of the second vectors in circles with different preset radii. This allows for a more comprehensive measurement of the displacement generated by non-linear motion, thus improving the detection accuracy of video image stabilization. Furthermore, the circles with different preset radii also have preset weight information. Based on this weight information, the displacement of the relevant parts can be measured in detail, offering high flexibility.

[0117] In one embodiment, determining the average displacement of each second vector in the first frame image and the second frame image based on the weight information corresponding to each second vector, the preset transformation matrix, and the preset radius includes:

[0118] Multiply each of the second vectors by the preset transformation matrix to obtain a first value that maps each of the second vectors onto the second frame image;

[0119] Obtain the difference between the sum of each of the first values ​​and the sum of each of the second vectors;

[0120] Obtain the quotient of the difference and the number of the second vectors;

[0121] The average displacement is obtained by multiplying the quotient by the weight information corresponding to the preset radius.

[0122] In this embodiment, the first value obtained by multiplying each second vector in a circle with a preset radius by a preset transformation matrix is ​​the mapping value of each second vector in a circle with a preset radius on the second frame image. Since the second vector itself is essentially a vector after the first vector is rotated on the first frame image by a preset radius and a preset angle, the difference obtained by subtracting the sum of the first values ​​corresponding to each second vector from the sum of each second vector can be understood as the sum of the displacements of each second vector on the first frame image and the second frame image.

[0123] Dividing this difference by the number of second vectors (and the number corresponding to the rotation angles) and then multiplying it by the weight information corresponding to the preset radius yields the average displacement of each second vector within the circle of the preset radius in the first and second frames. This average displacement is a weighted average displacement. After obtaining the average displacement of each second vector within the circle of the preset radius in the first and second frames, displacement information can be determined based on these average displacements. For example, the maximum or minimum value among the different average displacements can be multiplied by the number of circles corresponding to the preset radius to obtain the displacement information.

[0124] In one embodiment, determining the displacement information based on the average displacement of the second vector within circles of different preset radii on the first frame image and the second frame image includes:

[0125] The sum of the average displacements of the second vector in the first frame image and the second frame image of circles with different preset radii is determined as the displacement information.

[0126] In this embodiment, the sum of the average displacements corresponding to the second vectors in circles with different preset radii is directly determined as the displacement information. The method by which this disclosure obtains the displacement information based on each second vector, a preset transformation matrix, and the weight information corresponding to the preset radius can be seen in the following formula (5):

[0127]

[0128] Where (dx, dy) represents displacement information, where dx is the displacement along the horizontal axis with rotation, and dy is the displacement along the vertical axis with rotation; P i The weight information corresponds to the circles with a preset radius; N is the number of circles; M is the number of rotation angles, which is also the number of second vectors; A is the aforementioned preset transformation matrix. This is the homogeneous form of the second vector v' corresponding to the circle with radius r, as shown in the following formula (6):

[0129]

[0130] It should be noted that, in the embodiments of this disclosure, if the weight information corresponding to different preset radii is the same, then the above formula (5) can be further simplified to the following formula (7):

[0131]

[0132] Where I is the identity matrix; since the weights corresponding to the preset radii are consistent, the contribution of circles with different radii to the displacement information is the same. The jitter of the center of the circle can also be used to characterize the displacement information, where O is the center of the circle. The jitter difference caused by rotation in different parts of the image can also be taken into account through the jitter of the center.

[0133] In this embodiment, the sum of average displacements is used as displacement information, which comprehensively considers the displacement of image content with different radii in the image, making the final image stabilization effect determined based on this displacement information more accurate.

[0134] Figure 4 This is a flowchart of a method for determining a preset transformation matrix in an embodiment of this disclosure, as follows: Figure 4 As shown, it includes the following steps:

[0135] S21. Perform corner detection on the first frame image and the second frame image to obtain the first corner point corresponding to the first frame image and the second corner point corresponding to the second frame image;

[0136] S22. Perform corner point matching on the first corner point and the second corner point, and calculate the preset transformation matrix based on the matched corner points.

[0137] In this embodiment, taking the calculation of a preset transformation matrix based on the first frame image and the second frame image as an example, this disclosure can calculate the transformation matrix based on the detected corner points during the calculation of the transformation matrix between the first frame image and the second frame image. This disclosure can employ algorithms such as SIFT, Harris, or FAST for corner point detection, and this embodiment does not impose any limitations on this.

[0138] In a general sense, corner points are extreme points, that is, points with particularly prominent attributes in a certain aspect. They are isolated points with the greatest or least intensity in certain attributes, the endpoints of line segments, etc. Corner points can effectively reduce the amount of data while preserving important features of images and graphics. Feature points, on the other hand, are a relatively abstract concept, such as edge points or points that are helpful in analyzing specific problems. Feature points may also include directional information that is invalid for calculating displacements with rotation in this disclosure. Therefore, this disclosure uses corner points to determine the transformation matrix, which can effectively improve processing speed and save computational resources.

[0139] In one embodiment, the step of performing corner point matching on the first corner point and the second corner point, and calculating the preset transformation matrix based on the matched corner points, includes:

[0140] Corner point matching is performed on the first corner point and the second corner point based on optical flow method to obtain preliminary matching corner points;

[0141] The preliminary matching corner points are filtered based on the Random Sampling Consensus (RANSAC) method to obtain the target matching corner points;

[0142] The preset transformation matrix is ​​calculated based on the target matching corner points.

[0143] In this embodiment, since the video image basically satisfies the assumptions of the optical flow method, and the optical flow method is also suitable for estimating image motion, this disclosure uses the optical flow method for corner matching. The assumptions of the optical flow method include the following:

[0144] 1) Constant brightness, meaning that the brightness of the same point does not change over time, and the brightness between adjacent frames is constant.

[0145] 2) Small motion, that is, the position does not change drastically with the change of time, the frame taking time of adjacent video frames is continuous, or the motion of objects between adjacent frames is relatively "small".

[0146] 3) Spatial consistency: points that are adjacent to each other in a scene are also adjacent points when projected onto the image, and the adjacent points have the same velocity, that is, the pixels of the same sub-image have the same motion.

[0147] This disclosure can calculate the preset transformation matrix after obtaining the initial matching corner points based on the optical flow method. However, in order to improve the accuracy of the preset transformation matrix calculation, this disclosure further uses the RANSAC method to filter the matching corner points to obtain the target matching corner points, and calculates the transformation matrix based on the target matching corner points.

[0148] In one embodiment, the step of filtering the preliminary matching corner points based on the Random Sampling Consensus (RANSAC) method to obtain target matching corner points includes:

[0149] The target matching corner is obtained by performing a preset K rounds of filtering on the initial matching corner based on the RANSAC method; wherein, K is a positive integer greater than 1, and the matching corner obtained in the previous filtering based on the RANSAC method is the corner to be filtered in the next execution of the RANSAC method.

[0150] In this embodiment, multiple screenings are performed based on the RANSAC method. Since the matching corner points obtained from the previous RANSAC method screening are the corner points to be screened in the next RANSAC method execution, it is analogous to picking good data from a pile of good data being faster and more accurate than picking good data from a pile of messy data. Therefore, the above-mentioned method of this disclosure can improve the screening speed and accuracy of target matching corner points.

[0151] The RANSAC method takes as input a set of observed data, a parameterized model that can interpret or adapt to the observed data, and some reliable parameters. RANSAC achieves its goal by iteratively selecting a random subset of the data, which is assumed to be inliers, and then re-estimating the model using all the assumed inliers. Inliers are defined as points that fit the model, and outliers are points that do not fit the model.

[0152] The following formula (8) is the method for calculating the number of RANSAC iterations K in this disclosure:

[0153]

[0154] Where p represents the probability that there are no outliers in the obtained model, q represents the probability that a point in the data is an outlier, and s represents the number of model parameters. In the embodiments of this disclosure, the RANSAC method is used to estimate the transformation matrix. As described in the above formula (1), the transformation matrix includes four parameters (s, θ, t). x t y ), because the value of s here is 4.

[0155] It should be noted that, in the embodiments of this disclosure, if a preset transformation matrix is ​​calculated using multiple sets of adjacent frame images in a video image, since the corner points of the two adjacent frame images are extracted and matched based on the corner points when calculating the preset transformation matrix of the first and second adjacent frame images, in order to reduce redundant calculations, the corner points of one frame image can be stored for the calculation of the transformation matrix of the next adjacent frame. For example, if there are 3 frames of images, the corner points of the first and second frames are calculated and the transformation matrix between the first and second frames is calculated for the first time. When calculating the transformation matrix for the second time, only the corner points of the third frame image need to be detected. This disclosure can use the corner points of the second frame image detected in the first detection and the corner points of the third frame image detected in the current detection to calculate the transformation matrix between the second and third frames. It can be understood that storing the calculation result of the next frame as the calculation result of the previous frame of the next adjacent frame can reduce the amount of calculation by about half and improve the processing speed. In one embodiment, the first frame image and the second frame image are preprocessed images.

[0156] In the embodiments of this disclosure, the first frame image and the second frame image are preprocessed images. Calculating the transformation matrix based on the preprocessed images can improve the accuracy of the transformation matrix calculation. This allows for a more accurate video stabilization effect when determining the video stabilization effect based on the displacement information with rotation calculated from the transformation matrix.

[0157] In one embodiment, the preprocessing includes at least one of the following:

[0158] Size normalization;

[0159] Grayscale processing;

[0160] Histogram equalization processing.

[0161] In this embodiment, preprocessing includes size normalization, which unifies videos of different sizes to the same size. This reduces computational load and makes the image stabilization of different videos comparable and valuable for comparison.

[0162] The preprocessing in this disclosure also includes grayscale processing, which can reduce the impact of color information between frames on corner point extraction, and can also reduce the amount of computation, thereby improving the processing speed.

[0163] The preprocessing in this disclosure also includes histogram equalization, which can reduce the impact of brightness information between frames on corner point extraction. Gray-scale histogram equalization can basically ensure that the brightness of the video is consistent over a period of time.

[0164] In one embodiment, determining the image stabilization effect of the video image based on the rotational displacement information includes:

[0165] The stabilization effect of the video image is determined based on the displacement information corresponding to each adjacent image frame in the video image.

[0166] As previously described, the electronic device of this disclosure can determine the video stabilization effect based on all adjacent frames in a video image, or it can determine the video stabilization effect based on specified adjacent frames. In this embodiment, this disclosure processes all adjacent image frames in a video image to obtain the displacement information of each adjacent image frame, and determines the video stabilization effect accordingly, which can improve the accuracy of stabilization effect detection.

[0167] Figure 5 This is a flowchart of a video detection method in an embodiment of the present disclosure. Figure 3 ,like Figure 5 As shown, in Figure 1 Based on this, step S12 includes the following:

[0168] S12a. Determine the calculated value based on the displacement information;

[0169] S12b. Determine the image stabilization effect of the video image based on the calculated value; wherein the calculated value is inversely proportional to the image stabilization effect, and the calculated value includes at least one of the following:

[0170] The mean value of displacement information corresponding to different adjacent images;

[0171] Standard deviation of displacement information corresponding to different adjacent images;

[0172] The median value of displacement information corresponding to different adjacent images.

[0173] In this embodiment, when determining the stabilization effect of a video image based on displacement information, various calculated values ​​can be determined based on the displacement information. For example, the calculated values ​​include the mean, standard deviation, and median of displacement information corresponding to different adjacent images. Of course, the maximum or minimum value can also be included. Since the smaller the displacement, the better the stabilization effect, the stabilization effect of the video image can be determined based on the above calculated values.

[0174] As shown in formula (5) above, in the displacement information with rotation, dx represents the displacement of the rotational motion in the horizontal axis direction, and dy represents the displacement of the rotational motion in the vertical axis direction. Taking the mean as an example, this disclosure can also determine the stabilization effect in the horizontal axis direction separately based on the mean of each dx value, or determine the stabilization effect in the vertical axis direction separately based on the mean of each dy value. Of course, the stabilization effect of the video can also be determined by combining the stabilization effect in the horizontal axis direction determined by the mean and the stabilization effect in the vertical axis direction. This disclosure does not limit this aspect. The median, maximum, or minimum values ​​of the displacement information corresponding to different adjacent images can all be obtained using the above method, which will not be elaborated here.

[0175] Furthermore, when determining the video stabilization effect based on the displacement information corresponding to each adjacent image frame in the video image, the larger the standard deviation of the displacement information of different adjacent images, the greater the difference in jitter between the frames in the video, and therefore the worse the stabilization effect. Conversely, the smaller the standard deviation, the smaller the difference in jitter between the frames in the video, and therefore the better the stabilization effect. It should be noted that, similar to the mean method, this disclosure can also determine the stabilization effect in the horizontal axis direction separately based on the standard deviation of each dx value, or determine the stabilization effect in the vertical axis direction separately based on the standard deviation of each dy value. Alternatively, the video stabilization effect can be determined by combining the stabilization effect in the horizontal axis direction determined by the standard deviation and the stabilization effect in the vertical axis direction. This disclosure does not limit the embodiments in this way.

[0176] Figure 6 This is a schematic diagram illustrating the principle of video detection according to an embodiment of the present disclosure, such as... Figure 6As shown, after preprocessing the i-th frame (i.e., the first frame mentioned above) and the (i+1)-th frame (i.e., the second frame mentioned above) of the video image, including scaling, grayscale conversion, and histogram equalization, corner points are extracted. Then, optical flow tracking is performed using the optical flow method to obtain matching point pairs (i.e., preliminary matching corner points). For the preliminary matching corner points, a first affine transformation is performed, during which data is filtered, such as using the RANSAC method to obtain target matching corner points. Then, a second affine transformation is performed based on the target matching corner points to obtain the transformation matrix. Based on the obtained transformation matrix, the dithering (displacement with rotation) of adjacent frames is obtained using the circle drawing method, which allows for further analysis of the video's dithering.

[0177] Figure 7 This is a flowchart of a video detection method in an embodiment of the present disclosure. Figure 4 ,like Figure 7 As shown, in Figure 1 Based on this, the following are included before step S11:

[0178] S10. Receive the video image sent by the device that acquired the first frame image and the second frame image; the video image includes the first frame image and the second frame image;

[0179] After step S12, the following may be included:

[0180] S13. After determining the stabilization effect of the video image based on the displacement information, the stabilization effect is sent to the device.

[0181] In this embodiment, the video detection method can be applied to electronic devices other than servers or devices that acquire video images. For example, a device that acquires video images can upload the video to a server for analysis and processing. After processing and analyzing the uploaded video, the server can also send the analysis results back to the sender. For example, the video image received in step 20 can be a video image that marks the effective time period of the video and the region of interest (ROI).

[0182] Taking the server-side video detection method as an example, Figure 8 This is an example diagram of a video detection application scenario in an embodiment of this disclosure, such as... Figure 8As shown, after a user uploads a video to the server, the server saves the video to the cloud and returns a result containing a Message-Digest Algorithm (MD5) value to the user. The server analyzes the video using the video detection scheme mentioned in this paper, performs image stabilization detection, and saves the detection results to the cloud. While the server is executing the detection process, the user requests the video analysis results again using the MD5 value. The server receives this request; if it hasn't completed the video analysis, it responds to the request, informing the user to wait for the results. If the server has completed the video analysis and saved the results to the cloud, it sends a request to the cloud and retrieves the detection results, returning them to the user. The user can parse the analysis results and render them as charts for easy reading.

[0183] Figure 9 This is a diagram illustrating a video detection apparatus according to an exemplary embodiment. (Refer to...) Figure 9 Devices used in electronic devices include:

[0184] The acquisition module 101 is configured to acquire displacement information between adjacent first and second frame images in a video image; wherein, the displacement information includes displacement generated by the non-linear motion of the device acquiring the first and second frame images;

[0185] The determination module 102 is configured to determine the anti-shake effect of the video image based on the displacement information.

[0186] In some embodiments, the acquisition module 101 is further configured to determine a first vector based on a preset coordinate origin and first displacement information included in a preset transformation matrix; wherein the first vector represents the vector of the first displacement information relative to the preset coordinate origin, and the first displacement information is the translation information of pixels in the first frame image and the second frame image; rotate the first vector by different preset radii to obtain second vectors included in circles of each preset radius; wherein the directions of the second vectors in the circles of each preset radius are different; for each circle of preset radius, determine the average displacement of each second vector in the first frame image and the second frame image based on each second vector, the preset transformation matrix, and the weight information corresponding to the preset radius; wherein the weight information corresponding to the preset radius is used to represent the degree of attention to the image content in the first frame image; determine the displacement information based on the average displacement of the second vectors in the circles of different preset radii in the first frame image and the second frame image.

[0187] In some embodiments, the acquisition module 101 is further configured to multiply each of the second vectors by the preset transformation matrix to obtain a first value of each of the second vectors mapped onto the second frame image; obtain the difference between the sum of the first values ​​and the sum of the second vectors; obtain the quotient of the difference and the number of the second vectors; and multiply the quotient by the weight information corresponding to the preset radius as the average displacement.

[0188] In some embodiments, the acquisition module 101 is further configured to determine the displacement information as the sum of the average displacements of the second vectors in circles with different preset radii on the first frame image and the second frame image.

[0189] In some embodiments, the acquisition module 101 is further configured to determine the first vector based on the positional relationship between the preset coordinate origin and the center of the first frame image and the first displacement information if the preset coordinate origin is a point outside the center of the first frame image; wherein the first vector is the vector of the first displacement information relative to the center of the first frame image.

[0190] In some embodiments, the apparatus further includes:

[0191] The detection module 103 is configured to perform corner detection on the first frame image and the second frame image to obtain the first corner point corresponding to the first frame image and the second corner point corresponding to the second frame image;

[0192] The matching module 104 is configured to perform corner matching on the first corner point and the second corner point, and calculate the preset transformation matrix based on the matched corner points.

[0193] In some embodiments, the matching module 104 is further configured to perform corner matching on the first corner point and the second corner point based on optical flow to obtain preliminary matching corner points; filter the preliminary matching corner points based on the Random Sampling Consensus (RANSAC) method to obtain target matching corner points; and calculate the preset transformation matrix based on the target matching corner points.

[0194] In some embodiments, the matching module 104 is further configured to perform a preset K-round filtering on the initial matching corner points based on the RANSAC method to obtain the target matching corner points; wherein, K is a positive integer greater than 1, and the matching corner points obtained in the previous filtering based on the RANSAC method are the corner points to be filtered in the next execution of the RANSAC method.

[0195] In some embodiments, the determining module 102 is further configured to determine a calculated value based on the displacement information; and to determine the image stabilization effect of the video image based on the calculated value; wherein the calculated value is inversely proportional to the image stabilization effect, and the calculated value includes at least one of the following:

[0196] The mean value of displacement information corresponding to different adjacent images;

[0197] Standard deviation of displacement information corresponding to different adjacent images;

[0198] The median value of displacement information corresponding to different adjacent images.

[0199] In some embodiments, the apparatus further includes:

[0200] The receiving module 105 is configured to receive the video image sent by the device that acquired the first frame image and the second frame image; wherein the video image includes the first frame image and the second frame image;

[0201] The sending module 106 is configured to send the image stabilization effect to the device after determining the image stabilization effect of the video image based on the displacement information.

[0202] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0203] Figure 10 This is a block diagram illustrating a terminal device 800 according to an exemplary embodiment. For example, device 800 may be a mobile phone, mobile computer, etc.

[0204] Reference Figure 10 The device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0205] Processing component 802 typically controls the overall operation of device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0206] Memory 804 is configured to store various types of data to support the operation of device 800. Examples of this data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0207] The power supply component 806 provides power to the various components of the device 800. The power supply component 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 800.

[0208] Multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0209] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0210] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0211] Sensor assembly 814 includes one or more sensors for providing status assessments of various aspects of device 800. For example, sensor assembly 814 may detect the on / off state of device 800, the relative positioning of components such as the display and keypad of device 800, changes in the position of device 800 or a component of device 800, the presence or absence of user contact with device 800, the orientation or acceleration / deceleration of device 800, and temperature changes of device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0212] Communication component 816 is configured to facilitate wired or wireless communication between device 800 and other devices. Device 800 can access wireless networks based on communication standards, such as Wi-Fi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0213] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0214] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of the device 800 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0215] Figure 11 This is a block diagram illustrating a server apparatus 900 according to an exemplary embodiment. (Refer to...) Figure 11The apparatus 900 includes a processing component 922, which further includes one or more processors, and memory resources represented by memory 932 for storing instructions, such as application programs, that can be executed by the processing component 922. The application programs stored in memory 932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 922 is configured to execute instructions to perform the aforementioned network distribution method.

[0216] Device 900 may also include a power supply component 926 configured to perform power management of device 900, a wired or wireless network interface 950 configured to connect device 900 to a network, and an input / output (I / O) interface 958. Device 900 may operate on an operating system stored in memory 932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or similar.

[0217] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 932 including instructions, which can be executed by a processing component 922 of the apparatus 900 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0218] A non-transitory computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of the aforementioned terminal or server, enable the terminal or server to perform a video detection method, the method comprising:

[0219] The displacement information between adjacent first and second frames in a video image is obtained; wherein, the displacement information includes the displacement caused by the non-linear motion of the device acquiring the first and second frames;

[0220] The stabilization effect of the video image is determined based on the displacement information.

[0221] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0222] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A video detection method, characterized in that, The method includes: The displacement information between adjacent first and second frames in a video image is obtained; wherein, the displacement information includes the displacement caused by the non-linear motion of the device acquiring the first and second frames; The stabilization effect of the video image is determined based on the displacement information; The step of obtaining displacement information between adjacent first and second frame images in a video image includes: A first vector is determined based on the preset coordinate origin and the first displacement information included in the preset transformation matrix; wherein, the first vector is characterized as the vector of the first displacement information relative to the preset coordinate origin, and the first displacement information is the translation information of the pixels in the first frame image and the second frame image; The first vector is rotated by different preset radii to obtain a second vector included in a circle of each preset radius; wherein the directions of the second vectors in the circles of the preset radii are different. The average displacement of the second vector included in each preset radius circle on the first frame image and the second frame image is determined, and the displacement information is determined based on the average displacement of the second vector in circles with different preset radii.

2. The method according to claim 1, characterized in that, The average displacement of the second vector included in the circles with predetermined radii on the first frame image and the second frame image includes: For each circle with a preset radius, the average displacement of each second vector on the first frame image and the second frame image is determined based on each second vector, the preset transformation matrix, and the weight information corresponding to the preset radius; wherein, the weight information corresponding to the preset radius is used to characterize the degree of attention to the image content in the first frame image.

3. The method according to claim 2, characterized in that, The step of determining the average displacement of each second vector in the first frame image and the second frame image based on the weight information corresponding to each second vector, the preset transformation matrix, and the preset radius includes: Multiply each of the second vectors by the preset transformation matrix to obtain a first value that maps each of the second vectors onto the second frame image; Obtain the difference between the sum of each of the first values ​​and the sum of each of the second vectors; Obtain the quotient of the difference and the number of the second vectors; The average displacement is obtained by multiplying the quotient by the weight information corresponding to the preset radius.

4. The method according to claim 2, characterized in that, The displacement information is determined by the average displacement of the second vector within circles of different preset radii on the first and second frame images, including: The displacement information is determined by summing the average displacement of the second vector in circles with different preset radii on the first frame image and the second frame image.

5. The method according to claim 1, characterized in that, The step of determining the first vector based on the preset coordinate origin and the first displacement information included in the preset transformation matrix includes: If the preset coordinate origin is a point outside the center of the first frame image, the first vector is determined based on the positional relationship between the preset coordinate origin and the center of the first frame image and the first displacement information; wherein, the first vector is the vector of the first displacement information relative to the center of the first frame image.

6. The method according to claim 1, characterized in that, The method further includes: Corner detection is performed on the first frame image and the second frame image to obtain the first corner point corresponding to the first frame image and the second corner point corresponding to the second frame image; Corner point matching is performed on the first corner point and the second corner point, and the preset transformation matrix is ​​calculated based on the matched corner points.

7. The method according to claim 6, characterized in that, The step of performing corner point matching on the first corner point and the second corner point, and calculating the preset transformation matrix based on the matched corner points, includes: Corner point matching is performed on the first corner point and the second corner point based on optical flow method to obtain preliminary matching corner points; The preliminary matching corner points are filtered based on the Random Sampling Consensus (RANSAC) method to obtain the target matching corner points; The preset transformation matrix is ​​calculated based on the target matching corner points.

8. The method according to claim 7, characterized in that, The process of filtering the initial matching corner points using the Random Sampling Consensus (RANSAC) method to obtain target matching corner points includes: The target matching corner is obtained by performing a preset K rounds of filtering on the initial matching corner based on the RANSAC method; wherein, K is a positive integer greater than 1, and the matching corner obtained in the previous filtering based on the RANSAC method is the corner to be filtered in the next execution of the RANSAC method.

9. The method according to claim 1, characterized in that, Determining the image stabilization effect based on the displacement information includes: The calculated value is determined based on the displacement information; The image stabilization effect of the video image is determined based on the calculated value; wherein the calculated value is inversely proportional to the image stabilization effect, and the calculated value includes at least one of the following: The mean value of displacement information corresponding to different adjacent images; Standard deviation of displacement information corresponding to different adjacent images; The median value of displacement information corresponding to different adjacent images.

10. The method according to claim 1, characterized in that, The method further includes: The device receiving the video image sent by the device that acquired the first frame image and the second frame image; wherein the video image includes the first frame image and the second frame image; After determining the stabilization effect of the video image based on the displacement information, the stabilization effect is sent to the device.

11. A video detection device, characterized in that, The device includes: The acquisition module is configured to determine a first vector based on a preset coordinate origin and first displacement information included in a preset transformation matrix; wherein the first vector represents the vector of the first displacement information relative to the preset coordinate origin, and the first displacement information is the translation information of pixels in a first frame image and a second frame image; wherein the first frame image and the second frame image are adjacent frames in a video image; the first vector is rotated by different preset radii to obtain a second vector included in circles of each preset radius; wherein the directions of the second vectors in the circles of each preset radius are different; the average displacement of the second vectors included in the circles of each preset radius on the first frame image and the second frame image is determined, and displacement information is determined based on the average displacement of the second vectors in the circles of different preset radii; wherein the displacement information includes the displacement generated by the non-linear motion of the device acquiring the first frame image and the second frame image; The determination module is configured to determine the image stabilization effect of the video image based on the displacement information.

12. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to perform the video detection method as described in any one of claims 1 to 10.

13. A non-transitory computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the video detection method as described in any one of claims 1 to 10.

Images