Inter-frame video image stabilization method and device, computer equipment and storage medium

Abstract

The invention discloses an inter-frame video image stabilization method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring first feature point information; acquiring second feature point information based on pixel information in the first feature point information; determining a first motion offset according to the coordinate information of the first feature point information and the coordinate information of the second feature point information; if the first motion offset is greater than the preset offset, determining a target area containing a preset physical identifier corresponding to the first video frame in the second video frame; acquiring third feature point information of a preset physical identifier in the target area; determining a second motion offset of the first video frame and the second video frame according to the coordinate information of the first feature point information and the coordinate information of the third feature point information; and obtaining an image-stabilized video frame corresponding to the second video frame based on the second motion offset. The method does not need any hardware facility improvement, and can improve the image quality of the video frame in the video to be subjected to image stabilization.

Description

Technical Field

[0001] The present invention relates to the technical field of automobile safety testing, and in particular to an inter-frame video stabilization method, device, computer equipment and storage medium. Background Art

[0002] As the number of cars increases, the safety performance of cars is receiving more and more attention. Simulated collision test (slide table test) is an important test method for studying car safety. Compared with real car collision test, simulated collision test can develop and verify car safety components and systems more quickly, and its application in car safety development is becoming more and more extensive.

[0003] During the simulated collision test, a high-speed camera is needed to capture the details of the collision process, such as the movement posture of the collision dummy and the deployment of the airbag. The high-speed camera needs to be fixed on the slide to move together. However, due to the huge impact force generated during the collision, it is very easy to cause the high-speed camera to shake. This unexpected shaking will cause the video image obtained through the collision test to have serious shaking, making the image features in the video disordered and the image quality poor. It cannot meet the requirements of complete and clear image information acquisition, seriously affecting the feasibility and accuracy of subsequent image analysis, thereby affecting the collision test verification effect and the development of automobile safety test technology, causing significant economic losses. Therefore, it is necessary to stabilize the video to improve the image quality in the video.

[0004] In the prior art, video stabilization technology is divided into mechanical stabilization, optical stabilization and electronic stabilization. Due to the strong impact force generated during a car collision, for mechanical stabilization, even if the mechanical structure design is redundant, it cannot guarantee that the video shot will be completely free of jitter. For optical stabilization, the commonly used optical image stabilization optimization technology cannot be used in high-speed cameras, which are professional impact-resistant equipment. For electronic stabilization, since the information of each video frame is crucial in the analysis of car collision videos, the shooting frame rate of high-speed cameras is generally required to be above 1000FPS (Frames Per Second). Although the common electronic stabilization algorithm visually feels that the jitter has been eliminated, in fact, when watching frame by frame, there will be a phenomenon that the de-jitter effect of a certain frame is poor or even frame loss. In summary, the image features of the video to be stabilized obtained during the collision test are disordered and the image quality is poor. It has always been a professional problem in the field of automobile safety. How to stabilize the video to be stabilized to improve the image quality of the video to be stabilized is an urgent problem to be solved. Summary of the invention

[0005] The embodiments of the present invention provide an inter-frame video stabilization method, apparatus, computer equipment and storage medium to solve the problem of how to stabilize a video to be stabilized so as to improve the image quality of the video to be stabilized.

[0006] An inter-frame video stabilization method, comprising:

[0007] Acquire first feature point information of a preset physical marker of a first video frame, wherein the first feature point information includes coordinate information and pixel information of the preset physical marker;

[0008] Based on the pixel information in the first feature point information, feature matching is performed on the second video frame to obtain second feature point information of a corresponding preset physical marker in the second video frame, where the second feature point information includes coordinate information of the preset physical marker;

[0009] Determine a first motion offset of the first video frame and the second video frame according to the coordinate information of the first feature point information and the coordinate information of the second feature point information;

[0010] If the first motion offset is greater than a preset offset, determining a target area including a preset physical marker corresponding to the first video frame in the second video frame;

[0011] Acquire third feature point information of a preset physical marker in the target area, wherein the third feature point information includes coordinate information of the preset physical marker;

[0012] Determine a second motion offset of the first video frame and the second video frame according to the coordinate information of the first feature point information and the coordinate information of the third feature point information;

[0013] Based on the second motion offset, inverse motion compensation is performed on the second video frame to obtain a stabilized video frame corresponding to the second video frame.

[0014] Preferably, the number of the preset physical identifiers is two.

[0015] Preferably, after determining the first motion offset of the first video frame and the second video frame according to the coordinate information of the first feature point information and the coordinate information of the second feature point information, the inter-frame video stabilization method further includes:

[0016] If the first motion offset is not greater than a preset offset, inverse motion compensation is performed on the second video frame based on the first motion offset to obtain a stabilized video frame corresponding to the second video frame.

[0017] Preferably, after acquiring the stabilized video frame corresponding to the second video frame, the inter-frame video stabilization method further comprises:

[0018] Performing peak signal-to-noise ratio calculation based on the local area including the preset physical marker in the first video frame and the local area including the preset physical marker in the second video frame to determine a first peak signal-to-noise ratio value;

[0019] Determine a second peak signal-to-noise ratio value by performing peak signal-to-noise ratio calculation based on a local area including the preset physical marker in the stabilized video frame corresponding to the first video frame and a local area including the preset physical marker in the stabilized video frame corresponding to the second video frame;

[0020] An image stabilization effect evaluation of the image stabilized video frame is determined based on the first peak signal-to-noise ratio value and the second peak signal-to-noise ratio value.

[0021] Preferably, the first motion offset comprises a first position offset and a first rotation angle;

[0022] Preferably, if the first motion offset is greater than a preset offset, determining a target area including a preset physical marker corresponding to the first video frame in the second video frame comprises:

[0023] If the first position offset is greater than a position offset threshold, or the first rotation angle is greater than a rotation angle threshold, a target area including a preset physical mark of the first video frame is determined in the second video frame.

[0024] Preferably, the second motion offset comprises a second position offset and a second rotation angle;

[0025] The performing inverse motion compensation on the second video frame based on the second motion offset to obtain a stabilized video frame corresponding to the second video frame includes:

[0026] Based on the second position offset, performing position offset compensation on the second video frame;

[0027] Based on the second rotation angle, performing rotation angle compensation on the second video frame;

[0028] The stabilized video frame is a video frame obtained by performing position offset compensation and / or rotation angle compensation on the second video frame.

[0029] An inter-frame video stabilization device, comprising:

[0030] A first feature point information acquisition module, used to acquire first feature point information of a preset physical marker of a first video frame, wherein the first feature point information includes coordinate information and pixel information of the preset physical marker;

[0031] A second feature point information acquisition module, based on the pixel information in the first feature point information, performs feature matching on the second video frame to acquire second feature point information of a corresponding preset physical marker in the second video frame, wherein the second feature point information includes coordinate information of the preset physical marker;

[0032] A first motion offset determination module, used to determine a first motion offset of the first video frame and the second video frame according to the coordinate information of the first feature point information and the coordinate information of the second feature point information;

[0033] a target region determining module, configured to determine, in the second video frame, a target region including a preset physical marker corresponding to the first video frame if the first motion offset is greater than a preset offset;

[0034] A third feature point information acquisition module, used to acquire third feature point information of a preset physical marker in the target area, wherein the third feature point information includes coordinate information of the preset physical marker;

[0035] A second motion offset determination module, used to determine a second motion offset of the first video frame and the second video frame according to the coordinate information of the first feature point information and the coordinate information of the third feature point information;

[0036] The first stabilized video frame acquisition module performs inverse motion compensation on the second video frame based on the second motion offset to acquire a stabilized video frame corresponding to the second video frame.

[0037] Preferably, the inter-frame video stabilization device further comprises:

[0038] A first peak signal-to-noise ratio value determination module, which performs peak signal-to-noise ratio calculation based on a local area including a preset physical marker in the first video frame and a local area including the preset physical marker in the second video frame to determine a first peak signal-to-noise ratio value;

[0039] a second peak signal-to-noise ratio value determining module, performing peak signal-to-noise ratio calculation based on a local area including the preset physical marker in the stabilized video frame corresponding to the first video frame and a local area including the preset physical marker in the stabilized video frame corresponding to the second video frame, to determine a second peak signal-to-noise ratio value;

[0040] The image stabilization effect evaluation module determines an image stabilization effect evaluation of the image stabilized video frame based on the first peak signal-to-noise ratio value and the second peak signal-to-noise ratio value.

[0041] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned inter-frame video stabilization method when executing the computer program.

[0042] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned inter-frame video stabilization method is implemented.

[0043] The above-mentioned inter-frame video stabilization method, device, computer equipment and storage medium perform feature matching on the second video frame based on the first feature information of the first video frame to obtain the second feature point information corresponding to the second video frame. The feature matching method makes the feature matching more comprehensive and more robust; based on the first feature point information and the second feature point information, the first motion offset of the first video frame and the second video frame is determined more accurately; based on whether the first motion offset is greater than the preset offset, it is judged whether the feature matching result conforms to the actual situation. When the feature matching result does not conform to the actual situation, a target area of ​​a preset physical mark is determined on the second video frame by an artificial determination method. The method of performing local search in the target area in the second video frame obtains the third feature point information, improves the accuracy and efficiency of feature matching, and thus ensures that the subsequent stabilization motion compensation has a high accuracy; according to the first feature point information and the third feature point information, the second motion offset of the first video frame and the second video frame is determined, so that the obtained second motion offset is more accurate; based on the second motion offset, the reverse motion compensation operation can be performed to achieve image stabilization processing of the video to be stabilized, so that each stabilized video frame corresponding to the second video frame in the video to be stabilized has a good image quality, and the purpose of improving the image quality of each video frame in the video to be stabilized is achieved. In addition, the method can better stabilize the video to be stabilized without any hardware facility improvement, effectively reduce the collision jitter in the stabilized video frame after stabilization, and improve the image quality of each second video frame in the video to be stabilized, and it is low-cost, easy to operate, and saves hardware costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0045] Figure 1 is a flow chart of an inter-frame video stabilization method according to an embodiment of the present invention;

[0046] Figure 2 is another flow chart of an inter-frame video stabilization method according to an embodiment of the present invention;

[0047] Figure 3 is another flow chart of an inter-frame video stabilization method according to an embodiment of the present invention;

[0048] Figure 4 is a flow chart of an inter-frame video stabilization device in one embodiment of the present invention;

[0049] Figure 5is a schematic diagram of a computer device in one embodiment of the present invention;

[0050] Figure 6 It is a common preset physical mark in automobile collision tests in this field;

[0051] Figure 7 It is a schematic diagram of image positions of a first video frame and a second video frame. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] The video stabilization method provided by the embodiment of the present invention is specifically applied in a video stabilization system, which can be applied in a computer device to implement stabilization of a video to be stabilized, so as to improve the image quality of the video to be stabilized.

[0054] In one embodiment, if Figure 1 As shown, a video stabilization method is provided, and the method is applied in Figure 5 The computer device in the example is used as an example to illustrate, including the following steps:

[0055] S101: Acquire first feature point information of a preset physical marker of a first video frame, where the first feature point information includes coordinate information and pixel information of the preset physical marker;

[0056] S102: performing feature matching on the second video frame based on pixel information in the first feature point information, and obtaining second feature point information of a corresponding preset physical marker in the second video frame, where the second feature point information includes coordinate information of the preset physical marker;

[0057] S103: determining a first motion offset of the first video frame and the second video frame according to the coordinate information of the first feature point information and the coordinate information of the second feature point information;

[0058] S104: if the first motion offset is greater than a preset offset, determining a target area including a preset physical marker corresponding to the first video frame in the second video frame;

[0059] S105: Acquire third feature point information of a preset physical marker in the target area, where the third feature point information includes coordinate information of the preset physical marker;

[0060] S106: determining a second motion offset of the first video frame and the second video frame according to the coordinate information of the first feature point information and the coordinate information of the third feature point information;

[0061] S107: Based on the second motion offset, perform inverse motion compensation on the second video frame to obtain a stabilized video frame corresponding to the second video frame.

[0062] Among them, the first video frame refers to the video frame shot in front in the video to be stabilized. The second video frame refers to the video frame shot in the back in the video to be stabilized. Among them, the video to be stabilized refers to the video that needs to be stabilized. For example, the video to be stabilized refers to the video collected during the car collision test, including the entire process of the collision. The preset physical marker refers to a marker that is pre-pasted on a specific area of ​​the car before the collision test is carried out, which is used to enhance the image features and facilitate the subsequent feature matching according to the preset physical marker of each video frame in the video to be stabilized. In this example, the Marker marker is selected as the preset physical marker. It is understandable that before carrying out the collision test, it is necessary to post the Marker marker at a specific position of the car, so that after the video to be stabilized is collected later, the corresponding first feature point information and second feature point information are obtained according to the Marker marker, feature matching and video stabilization are performed, and the purpose of improving the image quality of the second video frame is achieved.

[0063] Among them, the first feature point information refers to the information corresponding to the image area containing the preset physical identifier determined in the first video frame. The first feature point information includes the coordinate information and pixel information of the preset physical identifier. The coordinate information here can be the coordinate information of the center point of the image area, or it can be the coordinate information of a corner point in the image area, for example, the coordinate information of the upper left corner of the rectangular area defined by the preset physical identifier. The pixel information here refers to the information representing all pixel points in the image area of ​​the preset physical identifier, including but not limited to the pixel grayscale values ​​of all pixel points.

[0064] As an example, in step S101, the computer device obtains a video to be stabilized containing a preset physical marker in a car collision test. The video to be stabilized includes a first video frame and a second video frame taken successively. After obtaining the video to be stabilized with the preset physical marker, the computer device obtains an image area containing the preset physical marker in the first video frame of the video to be stabilized, and obtains first feature point information including coordinate information and pixel information of the preset physical marker in the image area. Figure 6Common preset physical marks in automobile collision tests in the art are shown. The preset physical marks are mainly posted on the B-pillars, door frames, and other locations of the automobile where deformation is small during collision tests. Understandably, in collision tests, small deformation indicates that the location will produce small jitters when subjected to collision impact. Therefore, the features of the area containing the preset physical marks in each video frame of the video to be stabilized are relatively clear and orderly, which facilitates the subsequent more accurate feature matching of each second video frame of the video to be stabilized, and avoids the discrete invalidity of feature matching. In this example, the video to be stabilized includes multiple video frames within a period of time in the automobile collision test. In order to facilitate a clearer description of the stabilization process of each video frame in the video to be stabilized, two adjacent video frames in the video to be stabilized are selected as description objects, and the video frame shot in front is taken as the first video frame, and the video frame shot in the back is taken as the second video frame. Understandably, the first video frame and the second video frame can be any two video frames in the video to be stabilized that are continuous in shooting time. In this example, since the preset physical marker is obviously different from the environmental features, it is convenient to enhance the image features. The first feature point information including the coordinate information and pixel information of the preset physical marker in the acquired first video frame can have stronger image feature information, which is convenient to improve the accuracy of subsequent feature matching of the second video frame based on the first feature point information.

[0065] The second feature point information refers to feature point information that matches the pixel information in the first feature point information and is obtained by performing global feature matching in the second video frame based on the pixel information in the first feature point information. Since the first feature point information refers to information corresponding to the image area containing the preset physical identifier determined in the first video frame, the second feature point information matched based on the first feature point information is information corresponding to the image area containing the preset physical identifier matched in the second video frame, and the second feature point information includes coordinate information of the preset physical identifier, where the coordinate information can be the coordinate information of the center point of the image area, or can be the coordinate information of a corner point in the image area.

[0066] As an example, in step S102, the computer device performs feature matching on the second video frame shot later based on the pixel information in the first feature point information determined based on the first video frame, obtains feature point information in the second video frame that matches the pixel information in the first feature point information, and uses the feature point information as the second feature point information. The second feature point information includes the coordinate information of the preset physical marker in the second video frame.

[0067] For example, in the video to be stabilized, it is necessary to start from the first video frame, take the next video frame as the second video frame, perform feature matching on the second video frame according to the pixel information in the first feature point information corresponding to the first video frame, and obtain the second feature point information corresponding to the second video frame, so as to facilitate the subsequent acquisition of the first motion offset according to the first feature point information and the second feature point information. Block matching methods, optical flow algorithms, SIFT (scale invariant feature transform) algorithms, etc. can be used, based on the pixel information of the first feature point information in the first video frame, a global search in the second video frame, feature matching of each pixel information in the second video frame with the pixel information of the first feature point information, and obtaining information in the second video frame that matches the pixel information of the first feature point information, and the information is used as the second feature point information. In this example, of the two video frames captured at adjacent moments, the video frame shot first is determined as the first video frame, and the video frame shot later is determined as the second video frame. Feature matching is performed on the second video frame based on the pixel information in the first feature point information corresponding to the first video frame, and the second feature point information corresponding to the second video frame is obtained. And so on, frame-by-frame feature matching is performed on the stabilized video to obtain the second feature point information corresponding to each second video frame.

[0068] In this example, based on the pixel information in the first feature point information, feature matching is performed on the second video frame to obtain the second feature point information of the corresponding preset physical marker in the second video frame. This feature matching method makes the feature matching more comprehensive and robust, which facilitates the subsequent determination of the first motion offset based on the second feature point information to determine whether the second feature point information obtained by feature matching has a higher degree of accuracy.

[0069] The first motion offset is the offset between the coordinate information of the second feature point information corresponding to the second video frame and the coordinate information of the first feature point information corresponding to the first video frame. When the video to be stabilized is a video collected during a car collision test, the first motion offset can reflect the offset corresponding to the preset physical mark between two adjacent video frames (the first video frame and the second video frame).

[0070] As an example, in step S103, after determining that the second feature point information corresponding to the second video frame is matched, the computer device obtains the offset between the coordinate information of the first feature point information and the coordinate information of the second feature point information based on the second feature point information corresponding to the second video frame and the first feature point information corresponding to the first video frame, and uses the offset between the coordinate information of the first feature point information and the coordinate information of the second feature point information as the first motion offset between the first video frame and the second video frame. Understandably, Figure 7A schematic diagram of the image position of the first video frame and the second video frame is shown, wherein Figure (a) is a schematic diagram of the image position of the first video frame, and Figure (b) is a schematic diagram of the image position of the second video frame. The figure shows that there is a certain motion offset between the area corresponding to the first feature point information and the area corresponding to the second feature point information. Since the first video frame and the second video frame are adjacent video frames, if the jitter caused by the collision is small, the first motion offset between the first video frame and the second video frame is not large, and the offset between the coordinate information of the first feature point information and the coordinate information of the second feature point information is also not large. Conversely, if the jitter caused by the collision is large, the first motion offset between the first video frame and the second video frame is large, and the offset between the coordinate information of the first feature point information and the coordinate information of the second feature point information is also large. Therefore, the offset between the coordinate information of the first feature point information and the coordinate information of the second feature point information can be calculated through the coordinate information of the first feature point information and the coordinate information of the second feature point information, and the offset between the coordinate information of the first feature point information and the coordinate information of the second feature point information can be used as the first motion offset between the first video frame and the second video frame to evaluate whether the jitter between the two frames of video is more obvious, so as to determine whether the second video frame needs to be stabilized. In this example, the first motion offset of the first video frame and the second video frame is determined more accurately based on the first feature point information and the second feature point information.

[0071] Among them, the preset offset is a pre-set offset, which is used to evaluate whether there is an abnormality in the motion offset between two video frames. Understandably, due to the existence of a collision, collision splashes, occlusion, and / or the falling off of a preset physical mark, etc., the preset physical mark in the second video frame will change or disappear. At this time, the second feature point information obtained by matching the pixel information based on the first feature point information will be inaccurate, the error will be too large, and the feature matching will be abnormal, which is specifically manifested as the first motion offset being too large. Therefore, it is necessary to set a preset offset, and judge whether the first motion offset is too large by the preset offset, so as to determine whether the matching of the second feature point information is accurate, and then determine whether the feature matching is abnormal. If the first motion offset is greater than the preset offset, it indicates that the second feature point information matched based on the pixel value of the first feature point information is inaccurate, the error is too large, and the feature matching is abnormal; if the first motion offset is greater than the preset offset, it indicates that the second feature point information matched based on the pixel value of the first feature point information is relatively accurate, and the feature matching is normal.

[0072] As an example, in step S104, when the computer device determines that the first motion offset is greater than the preset offset, it can determine that the first motion offset between the two frames of video before and after is large, that is, the accuracy of the feature matching between the first feature point information and the second feature point information is not high. If the second video frame is directly subjected to reverse motion compensation based on the first motion offset, it will lead to poor image stabilization. Therefore, it is necessary to manually determine the target area containing the preset physical mark in the second video frame to facilitate the subsequent more accurate feature matching in the target area based on the first feature point information. In this example, whether the feature matching result conforms to the actual situation is determined based on whether the first motion offset is greater than the preset offset. When the feature matching result does not conform to the actual situation, the target area of ​​the preset physical mark is determined on the second video frame by manual determination, so that the third feature point information of the preset physical mark in the target area obtained later is more accurate, thereby ensuring the subsequent image stabilization accuracy of the second video frame.

[0073] The third feature point information refers to features obtained by performing local feature matching in the target area based on the first feature point information and matching the first feature point information.

[0074] As an example, in step S105, after obtaining the manually selected target area, the computer device performs a local search on the target area in the second video frame based on the first feature point information extracted in the first video frame, obtains feature point information that matches the first feature point information, uses the feature point information as the third feature point information obtained by re-matching, and obtains the coordinate information of the preset physical identifier included in the third feature point information. Among them, the process of performing a local search on the target area to obtain feature point information that matches the first feature point information can adopt a local block matching method, an optical flow algorithm, or a SIFT (scale invariant feature transform) algorithm. In this example, the method of performing a local search on the target area in the second video frame is used to obtain the third feature point information, thereby improving the accuracy and efficiency of feature matching, thereby ensuring that the subsequent stabilization motion compensation has a higher accuracy.

[0075] As an example, in step S106, the offset between the coordinate information of the first feature point information and the coordinate information of the third feature point information is determined based on the coordinate information of the first feature point information and the coordinate information of the third feature point information, and the offset between the coordinate information of the first feature point information and the coordinate information of the third feature point information is used as the second motion offset of the first video frame and the second video frame, and the size relationship between the second motion offset and the preset offset is determined. If the second motion offset is greater than the preset offset, continue to perform steps S104-S106 until the obtained second motion offset has a high degree of accuracy and meets the requirements of the preset offset, that is, the second motion offset is not greater than the preset offset. In this example, the second motion offset of the first video frame and the second video frame is determined based on the first feature point information and the third feature point information, so that the obtained second motion offset is more accurate, which is convenient for the subsequent motion compensation of the second video frame based on the second motion offset to obtain a more accurate stabilized video frame.

[0076] Understandably, after determining that the second motion offset is not greater than the preset offset, the second motion offset between the coordinate information of the first feature point information and the coordinate information of the third feature point information is obtained. The method determines the second motion offset only when the second motion offset is not greater than the preset offset.

[0077] The inverse motion compensation refers to performing an inverse motion compensation operation on the second video frame according to the second motion offset. The stabilized video frame is a video frame obtained by performing inverse motion compensation on the second video frame in the stabilized video.

[0078] As an example, in step S107, after determining the second motion offset, the computer device performs reverse motion compensation on the second video frame based on the second motion offset to obtain a stabilized video frame. Understandably, if the position offset between the coordinate information of the first feature point information and the coordinate information of the third feature point information is △X and △Y, and the rotation angle is α, then each position area in the second video frame is reversely translated by △X and △Y, and reversely rotated by angle α, to implement the operation process of reverse motion compensation on the second video frame, and obtain a stabilized video frame after reverse motion compensation.

[0079] For example, the computer device starts to perform stabilization processing from the first video frame of the video to be stabilized, takes the first video frame of the video to be stabilized as the first video frame, takes the second video frame of the video to be stabilized as the second video frame, executes the above steps S101 to S106, after determining the offset between the coordinate information corresponding to the first feature point information and the coordinate information corresponding to the third feature point information, and taking the offset between the coordinate information corresponding to the first feature point information and the coordinate information corresponding to the third feature point information as the second motion offset, based on the second motion offset, performs reverse motion compensation on the second video frame to achieve stabilization processing on the second video frame, obtains the corresponding stabilized video frame, and achieves stabilization processing on the second video frame in the video to be stabilized. Subsequently, the acquired stabilized video frame is used as the first video frame, and the third video frame of the video to be stabilized is used as the second video frame. After determining the first video frame and the second video frame again, continue to execute the above steps S101 to S106. After determining the offset between the coordinate information corresponding to the first feature point information and the coordinate information corresponding to the third feature point information, and taking the offset between the coordinate information corresponding to the first feature point information and the coordinate information corresponding to the third feature point information as the second motion offset, perform reverse motion compensation on the second video frame based on the second motion offset to obtain the corresponding stabilized video frame, thereby achieving stabilization processing of the third video frame in the video to be stabilized, and so on, to obtain the stabilized video frame corresponding to each second video frame in the video to be stabilized, thereby achieving the purpose of performing stabilization processing on each second video frame in the video to be stabilized and improving the image quality of each second video frame.

[0080] For another example, a computer device starts performing stabilization processing from the first video frame of the video to be stabilized, takes the first video frame of the video to be stabilized as the first video frame, takes the second video frame of the video to be stabilized as the second video frame, executes the above steps S101 to S106, after determining the offset between the coordinate information corresponding to the first feature point information and the coordinate information corresponding to the third feature point information, and taking the offset between the coordinate information corresponding to the first feature point information and the coordinate information corresponding to the third feature point information as the second motion offset, based on the second motion offset, performs reverse motion compensation on the second video frame to achieve stabilization processing on the second video frame, obtains the corresponding stabilized video frame, and achieves stabilization processing on the second video frame in the video to be stabilized. Subsequently, the acquired stabilized video frame is used as the first video frame, and the third feature point information corresponding to the stabilized video frame obtained by inverse motion compensation when the second video frame is used as the second video frame in the previous stabilization process is used as the first feature point information of this stabilization process. During this stabilization process, the first feature point information is no longer re-determined, and the third video frame in the video to be stabilized is used as the second video frame, and the above steps S102 to S105 are executed. After determining the second motion offset between the coordinate information of the first feature point information and the coordinate information of the third feature point information, inverse motion compensation is performed on the second video frame based on the second motion offset to achieve stabilization processing on the second video frame and obtain the corresponding stabilized video frame. By analogy, the stabilized video frame corresponding to each second video frame in the video to be stabilized is obtained, so as to achieve the purpose of performing stabilization processing on each second video frame in the video to be stabilized and improving the image quality of each second video frame.

[0081] In this example, the reverse motion compensation operation is performed based on the second motion offset, which can achieve stabilization processing of the video to be stabilized, so that the stabilized video frames corresponding to each second video frame in the video to be stabilized have good image quality, thereby achieving the purpose of improving the image quality of each video frame in the video to be stabilized.

[0082] In this embodiment, feature matching is performed on the second video frame based on the first feature information of the first video frame to obtain second feature point information corresponding to the second video frame. The feature matching method makes the feature matching more comprehensive and more robust. The first motion offset of the first video frame and the second video frame is determined more accurately according to the first feature point information and the second feature point information. Whether the feature matching result conforms to the actual situation is judged according to whether the first motion offset is greater than the preset offset. When the feature matching result does not conform to the actual situation, the target area of ​​the preset physical mark is determined on the second video frame by an artificial determination method, and the third feature point information is obtained by a local search method in the target area of ​​the second video frame, so as to improve the accuracy and efficiency of feature matching, thereby ensuring that the subsequent stabilization motion compensation has a higher accuracy. The second motion offset of the first video frame and the second video frame is determined according to the first feature point information and the third feature point information, so that the obtained second motion offset is more accurate. The reverse motion compensation operation is performed based on the second motion offset, so that the image stabilization processing of the video to be stabilized can be realized, so that the stabilized video frame corresponding to each second video frame in the video to be stabilized has a good image quality, and the purpose of improving the image quality of each video frame in the video to be stabilized is achieved. Moreover, this method can better stabilize the video to be stabilized without any improvement in hardware facilities, effectively reduce the collision jitter in the stabilized video frame after stabilization, and improve the image quality of each second video frame in the video to be stabilized. It is also low-cost, easy to operate, and saves hardware costs. It has important research significance in the early research and development experiments in the field of automobile safety test technology.

[0083] In one embodiment, the number of preset physical identifiers is two.

[0084] As an example, the number of preset physical identifiers in the first video frame obtained can be two. Among them, the two preset physical identifiers obtained in the first video frame define the matrix area, and the two preset physical identifiers are used to define a diagonal of the matrix area. It can be understood that in the video to be stabilized, the position distance of the two diagonal areas is far, and the farther the position distance, the better the compensation effect when performing motion compensation after the subsequent feature matching is successful. This method facilitates the subsequent more accurate stabilization processing of each second video frame of the video to be stabilized. For example, the preset physical identifier can select a Marker identifier, and after the computer device obtains the video to be stabilized, it reminds the tester to select two Marker identifiers in the first video frame of the video to be stabilized, and obtain the first feature point information corresponding to the coordinate information of the two Marker identifiers. A more preferred way to select two Marker identifiers is to use the first video frame in the video to be stabilized as the first video frame, and select two Marker identifiers of the diagonal area from the first video frame, so that the compensation effect of reverse motion compensation after subsequent matching is successful is better.

[0085] In another embodiment, after step S103, that is, after determining the first motion offset of the first video frame and the second video frame according to the coordinate information of the first feature point information and the coordinate information of the second feature point information, the inter-frame video stabilization method also includes: if the first motion offset is not greater than the preset offset, then based on the first motion offset, performing reverse motion compensation on the second video frame to obtain a stabilized video frame corresponding to the second video frame.

[0086] As an example, when the computer device determines that the first motion offset is not greater than the preset offset, it can determine that the first motion offset between the two frames of video before and after is small, that is, the accuracy of the feature matching between the first feature point information and the second feature point information is high, and the second video frame can be reversely motion compensated based on the first motion offset to obtain a stabilized video frame. Understandably, if in the first motion offset, the position offset is △X and △Y, and the rotation angle is α, then each position in the second video frame is reversely translated by △X and △Y, and reversely rotated by angle α, to implement the operation process of reverse motion compensation for the second video frame, and obtain a stabilized video frame after reverse motion compensation.

[0087] In this embodiment, when it is determined that the first motion offset is not greater than the preset offset, it can be determined that the motion offset between the two previous and next video frames is small and the matching accuracy is high. At this time, a reverse motion compensation operation is performed based on the first motion offset, which can ensure that the stabilized video frames corresponding to each second video frame in the video to be stabilized have good image quality, thereby achieving the purpose of improving the image quality of each video frame in the video to be stabilized.

[0088] In one embodiment, if Figure 2 As shown, after step S107, that is, after obtaining the stabilized video frame corresponding to the second video frame, the inter-frame video stabilization method further includes:

[0089] S201: Calculating a peak signal-to-noise ratio based on a local area including a preset physical marker in a first video frame and a local area including a preset physical marker in a second video frame to determine a first peak signal-to-noise ratio value;

[0090] S202: Calculating a peak signal-to-noise ratio based on a local area including a preset physical marker in a stabilized video frame corresponding to the first video frame and a local area including a preset physical marker in a stabilized video frame corresponding to the second video frame, to determine a second peak signal-to-noise ratio value;

[0091] S203: Determine an image stabilization effect evaluation of the image stabilized video frame based on the first peak signal-to-noise ratio value and the second peak signal-to-noise ratio value.

[0092] Among them, the preset physical mark refers to the preset physical mark selected in the first video frame for calculating the peak signal-to-noise ratio value. It is understandable that when shooting the collision test video, although the high-speed camera and the vehicle body are theoretically relatively still, the collision test video has complex images, such as dummies, test instruments, vehicle sheet metal, vehicle interior and other objects. Many objects move or break during the collision, and the image features are disordered. Some feature areas change or even disappear. In order to truly evaluate the video stabilization effect, the local area of ​​the vehicle body structure where the preset physical mark is posted is selected as the basis for calculating the peak signal-to-noise ratio value. Since the preset physical mark is mainly posted on the B-pillar, door frame and other locations of the car that are less deformed during the collision test, it is not necessarily deformed, moved or broken. Therefore, the local area containing the preset physical mark is relatively stable during the collision process, and theoretically it will be relatively still with the high-speed camera.

[0093] The first peak signal-to-noise ratio value refers to a peak signal-to-noise ratio value between a local area including a preset physical marker in the first video frame and a local area including a preset physical marker in the second video frame.

[0094] As an example, in step S201, the computer device may perform peak signal-to-noise ratio calculation on the local area including the preset physical mark in the first video frame and the local area including the preset physical mark in the second video frame, and determine the peak signal-to-noise ratio (PSNR) between the two as the first peak signal-to-noise ratio. For example, the local area of ​​the preset physical mark can be an area with a preset radius centered on the center point of the preset physical mark. The local area including the preset physical mark can be an area with a preset radius centered on the center point of the preset physical mark. For example, a local area with a preset radius of 3, centered at (X, Y), that is, (X-1, Y+1), (X, Y+1), (X+1, Y+1), (X-1, Y), (X, Y), (X+1, Y), (X-1, Y-1), (X, Y-1) and (X+1, Y-1) is used as a local area. In this example, the computer device obtains a first peak signal-to-noise ratio value between the first video frame and the second video frame based on the pixel information of each coordinate position of the local area including the preset physical mark in the first video frame and the pixel information of each coordinate position of the local area including the preset physical mark in the second video frame. Specifically, the calculation formula of the first peak signal-to-noise ratio value is as follows: in, Among them, I K1 is a local area including a preset physical mark in the first video frame of the video to be stabilized, I C1 is a local area including a preset physical mark in the second video frame of the video to be stabilized, PSNR (I K1 , IC1 ) represents the first peak signal-to-noise ratio value between the local area including the preset physical mark in the first video frame and the local area including the preset physical mark in the second video frame. RMSE(I K1 , I C1 ) represents the mean square error between the local area including the preset physical mark in the first video frame and the local area including the preset physical mark in the second video frame, and M and N are the height and width of the local area including the preset physical mark in the first video frame and the second video frame, respectively. K1 (x, y) is the pixel information of the corresponding coordinate position in the local area including the preset physical mark in the first video frame, G C1 In this example, the first peak signal-to-noise ratio value between the local area including the preset physical mark in the first video frame and the local area including the preset physical mark in the second video frame in the video to be stabilized is obtained, so as to facilitate the subsequent evaluation of the stabilization effect according to the first peak signal-to-noise ratio value.

[0095] Among them, the second peak signal-to-noise ratio value is the peak signal-to-noise ratio value between the local area including the preset physical identifier in the stabilized video frame corresponding to the first video frame and the local area including the preset physical identifier in the stabilized video frame corresponding to the second video frame. It can be understood that in order to evaluate the stabilization effect of the video frame stabilization, the stabilization effect is judged by comparing the peak signal-to-noise ratios before and after the video stabilization. The peak signal-to-noise ratio is usually calculated based on the area with the same preset physical identifier between adjacent video frames. When calculating the second peak signal-to-noise ratio after stabilization as a control variable, it is necessary to calculate the second peak signal-to-noise ratio in the video frame after stabilization based on the local area containing the same preset physical identifier in the first video frame and the second video frame before stabilization.

[0096] As an example, in step S202, the computer device obtains a local area ROI1 including a preset physical marker in the stabilized video frame corresponding to the first video frame, and a local area ROI2 including a preset physical marker in the stabilized video frame corresponding to the second video frame, and obtains a peak signal-to-noise ratio value between ROI1 and ROI2, and uses the peak signal-to-noise ratio value as the second peak signal-to-noise ratio value. In this example, based on the pixel information of each coordinate position in the local area ROI1 including the preset physical marker in the stabilized video frame corresponding to the first video frame and the pixel information of each coordinate position in the local area ROI2 including the preset physical marker in the stabilized video frame corresponding to the second video frame, the second peak signal-to-noise ratio value between the local area including the preset physical marker in the first video frame and the local area including the preset physical marker in the second video frame is obtained. Specifically, the second peak signal-to-noise ratio value calculation formula is as follows: in, Among them, I K2 is a local area ROI1 including a preset physical mark in the stabilized video frame of the first video frame, C2 is a local area ROI2 including a preset physical marker in the stabilized video frame of the second video frame, PSNR(I K2 , I C2 ) represents the second peak signal-to-noise ratio value between the local area ROI1 including the preset physical mark in the stabilized video frame of the first video frame and the local area ROI2 including the preset physical mark in the stabilized video frame of the second video frame. RMSE(I K2 , I C2 ) represents the mean square error between the local area ROI1 including the preset physical mark in the stabilized video frame of the first video frame and the local area ROI2 including the preset physical mark in the stabilized video frame of the second video frame, and M and N are the height and width of ROI1 and ROI2 respectively. K2 (x, y) is the pixel information of the coordinate position in the local area ROI1 including the preset physical mark in the first video frame, G C2 (x, y) is the pixel information of the coordinate position in the local area ROI2 including the preset physical mark in the second video frame. In this example, the local area peak signal-to-noise ratio is calculated based on the local area including the preset physical mark in the stabilized video frame corresponding to the first video frame and the local area including the preset physical mark in the stabilized video frame corresponding to the second video frame, and the second peak signal-to-noise ratio value is determined, so as to facilitate the subsequent evaluation of the stabilization effect according to the second peak signal-to-noise ratio value.

[0097] The image stabilization effect evaluation is an evaluation result of whether the image stabilization video frame corresponding to the second video frame has a good image stabilization effect.

[0098] As an example, in step S203, the computer device determines the image stabilization effect evaluation of the stabilized video frame based on the first peak signal-to-noise ratio value and the second peak signal-to-noise ratio value. In this example, the computer device obtains the first peak signal-to-noise ratio value of the second video frame and the second peak signal-to-noise ratio value of the stabilized video frame, and obtains the image stabilization effect evaluation corresponding to the stabilized video frame. In this example, the first peak signal-to-noise ratio value and the second peak signal-to-noise ratio value are used to scientifically quantify the image stabilization effect, and the image stabilization effect evaluation of the stabilized video frame is obtained based on the first peak signal-to-noise ratio value and the second peak signal-to-noise ratio value, so as to more directly and accurately obtain the image stabilization effect evaluation corresponding to the stabilized video frame obtained by performing image stabilization processing on the second video frame.

[0099] In this embodiment, when evaluating the image stabilization effect, a method of selecting a special local area of ​​the image is adopted to calculate the local peak signal-to-noise ratio value of two adjacent video frames as an evaluation index. Compared with the method of using the global peak signal-to-noise ratio value as the image stabilization evaluation index, this method does not consider the subjective movement of the moving objects in the image, and can more truly reflect the image stabilization effect and quality.

[0100] In another embodiment, step S203, i.e., determining the image stabilization effect evaluation of the image stabilized video frame based on the first peak signal-to-noise ratio value and the second peak signal-to-noise ratio value, includes:

[0101] S2031: If the second peak signal-to-noise ratio value is greater than the first peak signal-to-noise ratio value, it is determined that the image stabilization effect of the image stabilized video frame is better;

[0102] S2032: If the second peak signal-to-noise ratio value is not greater than the first peak signal-to-noise ratio value, it is determined that the image stabilization effect of the stabilized video frame is poor.

[0103] As an example, in step S2031, when the computer device determines that the second peak signal-to-noise ratio value is greater than the first peak signal-to-noise ratio value, it determines that the image stabilization effect of the stabilized video frame obtained by performing the image stabilization processing on the second video frame is better.

[0104] As an example, in step S2032, when the computer device determines that the second peak signal-to-noise ratio value corresponding to the second video frame is not greater than the first peak signal-to-noise ratio value, it determines that the image stabilization effect of the stabilized video frame obtained by performing image stabilization processing on the second video frame is poor.

[0105] It is understandable that when performing reverse motion compensation on the second video frame to obtain a stabilized video frame to solve the problem of disordered image features and poor image quality of the second video frame due to collision jitter, the smaller the peak signal-to-noise ratio value between the first video frame and the second video frame, the greater the collision jitter between the second video frame and the first video frame, and the larger the peak signal-to-noise ratio value between the first video frame and the second video frame, the smaller the collision jitter between the second video frame and the first video frame. Similarly, the smaller the peak signal-to-noise ratio value between the stabilized video frame of the second video frame and the stabilized video frame of the first video frame, the greater the collision jitter between the stabilized video frame of the second video frame and the stabilized video frame of the first video frame, and the larger the peak signal-to-noise ratio value between the stabilized video frame of the second video frame and the stabilized video frame of the first video frame, the smaller the collision jitter between the stabilized video frame of the second video frame and the stabilized video frame of the first video frame. Therefore, if the second peak signal-to-noise ratio value between the stabilized video frame of the second video frame and the stabilized video frame of the first video frame is greater than the first peak signal-to-noise ratio value between the second video frame and the first video frame, it indicates that the collision jitter between the stabilized video frame of the second video frame and the stabilized video frame of the first video frame is less than the collision jitter between the second video frame and the first video frame, and the stabilized video frame has a better stabilization effect. On the contrary, if the second peak signal-to-noise ratio value between the stabilized video frame of the second video frame and the stabilized video frame of the first video frame is not greater than the first peak signal-to-noise ratio value between the second video frame and the first video frame, it indicates that the collision jitter between the stabilized video frame of the second video frame and the stabilized video frame of the first video frame is not less than the collision jitter between the second video frame and the first video frame, and the stabilized video frame has a poor stabilization effect.

[0106] In this embodiment, a first peak signal-to-noise ratio value corresponding to the second video frame and a second peak signal-to-noise ratio value corresponding to the stabilized video frame are obtained respectively, and the image stabilization effect evaluation of the stabilized video frame is determined based on the first peak signal-to-noise ratio value and the second peak signal-to-noise ratio value. This method uses the peak signal-to-noise ratio value to scientifically quantify the image stabilization effect of the second video frame and the stabilized video frame, so as to more directly and accurately obtain the image stabilization effect evaluation of the stabilized video frame obtained by performing image stabilization processing on the second video frame.

[0107] In one embodiment, the first motion offset includes a first position offset and a first rotation angle. The first position offset refers to the offset of coordinate information between the coordinate information of the first feature point information and the coordinate information of the second feature point information. The first rotation angle refers to the offset of the angle between the first feature point information and the second feature point information.

[0108] In one embodiment, step S104, i.e., if the first motion offset is greater than the preset offset, determining a target area including a preset physical marker corresponding to the first video frame in the second video frame, includes:

[0109] If the first position offset is greater than the position offset threshold, or the first rotation angle is greater than the rotation angle threshold, a target area including a preset physical mark of the first video frame is determined in the second video frame.

[0110] The position offset threshold is used to determine the size of the first position offset, and is the threshold of the position offset between the coordinate information of the first feature point information and the coordinate information of the second feature point information. The rotation angle threshold is used to determine the size of the first rotation angle, and is the threshold of the rotation angle between the first feature point information and the second feature point information. The first offset threshold and the first rotation angle threshold are both non-negative numbers.

[0111] As an example, when the computer device determines that the position offset between the coordinate information of the first feature point information and the coordinate information of the second feature point information is greater than the position offset threshold, or the rotation angle between the first feature point information and the second feature point information is greater than the rotation angle threshold, it is determined that the jitter generated in the area corresponding to the second feature point information is large, and it is determined that the second feature point information matched by the feature is not accurate, and the feature matching is abnormal. In this example, the preset position offset threshold is T1, the preset rotation angle threshold is T2, the X-axis offset between the coordinate information of the first feature point information and the coordinate information of the second feature point information is △X, and the Y-axis offset is △Y, then the offset between the coordinate information of the first feature point information and the coordinate information of the second feature point information is The rotation angle between the first feature point information and the second feature point information is like or, It is determined that the second feature point information matched by the feature is not accurate, and the feature matching is abnormal. At this time, it is necessary to manually determine the target area containing the preset physical mark in the second video frame. This method of manually selecting the target area is convenient for ensuring the accuracy of image stabilization. In this example, when it is determined that the first position offset is greater than the position offset threshold, or the first rotation angle is greater than the rotation angle threshold, the target area containing the preset physical mark is determined in the second video frame, which is convenient for subsequent local feature matching in the target area based on the first feature point information to obtain the third feature point information.

[0112] Similarly, when the computer device determines that the position offset between the coordinate information of the first feature point information and the coordinate information of the second feature point information is not greater than the position offset threshold, and the rotation angle between the first feature point information and the second feature point information is not greater than the rotation angle threshold, it is determined that the jitter generated in the area corresponding to the second feature point information is small, and the second feature point information matched by the feature is determined to be more accurate, and the feature match is determined to be successful. In this example, the preset position offset threshold is T1, the preset rotation angle threshold is T2, the X-axis offset between the coordinate information of the first feature point information and the coordinate information of the second feature point information is △X, and the Y-axis offset is △Y, then the offset between the coordinate information of the first feature point information and the coordinate information of the second feature point information is The rotation angle between the first feature point information and the second feature point information is like and It is determined that the second feature point information obtained by feature matching is relatively accurate, and the feature matching is deemed successful. At this time, reverse motion compensation is directly performed on the second video frame based on the first motion offset to obtain a stabilized video frame corresponding to the second video frame.

[0113] For example, when the video to be stabilized is a collision test video, since the shooting frame rate of the collision test video is above 1000FPS (Frames Per Second), theoretically, the amount of motion between two consecutive video frames will not differ too much, that is, the position offset and rotation angle between the coordinate information of the first feature point information and the coordinate information of the second feature point information will not be too large. Therefore, if the position offset between the coordinate information of the first feature point information and the coordinate information of the second feature point information obtained by calculation exceeds the preset position offset threshold, or the rotation angle exceeds the preset rotation angle threshold, it means that the feature matching error of the second feature point information in the second video frame is too large and not accurate enough, and the matching of the second feature point information is determined to be abnormal. If the position offset between the coordinate information of the first feature point information and the coordinate information of the second feature point information obtained by calculation does not exceed the preset position offset threshold, and the rotation angle does not exceed the preset rotation angle threshold, it means that the matching of the second feature point information in the second video frame is accurate enough, and the second feature point information is determined as the final result of the feature matching.

[0114] In this embodiment, when it is determined that the first position offset is greater than the position offset threshold, or the first rotation angle is greater than the rotation angle threshold, a target area containing a preset physical marker is determined in the second video frame, so as to facilitate subsequent local feature matching in the target area based on the first feature point information, obtain the third feature point information, improve the feature matching accuracy, avoid the poor image quality of the stabilized video frame obtained by subsequent video stabilization based on the third feature point information due to inaccurate matching, and facilitate improving the image quality of the stabilized video frame.

[0115] In one embodiment, the second motion offset includes a second position offset and a second rotation angle. The second position offset refers to the offset of the position coordinates between the coordinate information of the first feature point information and the coordinate information of the third feature point information. The second rotation angle refers to the offset of the angle between the first feature point information and the third feature point information.

[0116] In one embodiment, if Figure 3 As shown, step S107, i.e., performing inverse motion compensation on the second video frame based on the second motion offset to obtain a stabilized video frame corresponding to the second video frame, includes:

[0117] S301: Based on the second position offset, compensating the position offset of the second video frame;

[0118] S302: performing rotation angle compensation on the second video frame based on the second rotation angle;

[0119] The stabilized video frame is a video frame obtained by performing position offset compensation and / or rotation angle compensation on the second video frame.

[0120] Among them, the second position offset compensation refers to adjusting the coordinate position of the second video frame in the opposite direction according to the position offset between the coordinate information of the third feature point information in the second video frame and the coordinate information of the first feature point information in the first video frame. The offset compensation controls the offset of the adjusted second video frame relative to the first video frame within a smaller range, and solves the problem of disordered image features in the second video frame due to collision jitter, and the problem of poor image quality in the second video frame due to collision jitter.

[0121] As an example, in step S301, after determining the second position offset between the third feature point information in the second video frame and the first feature point information in the first video frame, the computer device performs offset compensation on the second video frame based on the second position offset. Figure 7The schematic diagram of the image position of the first video frame and the second video frame is shown, wherein Figure (a) is a schematic diagram of the image position of the first video frame, and Figure (b) is a schematic diagram of the image position of the second video frame. If the position offset of the second video frame in Figure (b) relative to the first video frame in Figure (a) is Then the position offset of the second video frame on the X-axis relative to the first video frame is △X, and the position offset on the Y-axis is △Y. When performing position offset compensation, each position in the second video frame is moved in the opposite direction by △X on the X-axis and △Y on the Y-axis, so as to realize position offset compensation for the second video frame based on the first position offset. In this example, position offset compensation is performed on the second video frame based on the first position offset, so as to solve the problem of disordered images and poor image quality in the second video frame and improve the image quality in the second video frame.

[0122] The rotation angle compensation refers to adjusting the angle of the second video frame in the opposite direction according to the rotation angle between the third feature point information in the second video frame and the first feature point information in the first video frame. The rotation angle compensation makes the rotation angle of the adjusted second video frame relative to the first video frame controlled within a smaller range, solving the problem of disordered image features in the second video frame due to collision jitter and the problem of poor image quality in the second video frame due to collision jitter.

[0123] As an example, in step S302, after obtaining the second rotation angle, the computer device performs rotation angle compensation on the second video frame based on the second rotation angle. Figure 7 The schematic diagram of the image position between the first video frame and the second video frame is shown, FIG (a) is a schematic diagram of the image position of the first video frame, and FIG (b) is a schematic diagram of the image position of the second video frame. If the rotation angle of the second video frame in FIG (b) relative to the first video frame in FIG (a) is When performing rotation angle compensation, each position area in the second video frame is rotated in the opposite direction. The second video frame is compensated for the rotation angle based on the second rotation angle. In this example, the second video frame is compensated for the rotation angle based on the second rotation angle, so as to further solve the problem of image disorder and poor image quality in the second video frame and improve the image quality in the second video frame.

[0124] Understandably, due to various conditions of car collision, the jitter conditions of different video frames in the video to be stabilized are different. Therefore, the second video frame may only have a rotation angle relative to the first video frame, or may only have a position offset of the coordinate position, or may have both a rotation angle and a position offset of the coordinate position. Therefore, in this example, the stabilized video frame is a video frame after the second video frame is compensated for the offset and / or the rotation angle. That is, if there is only a rotation angle, the stabilized video frame is a video frame after the second video frame is compensated for the rotation angle; if there is only a position offset of the coordinate position, the stabilized video frame is a video frame after the second video frame is compensated for the position offset; if there is both a rotation angle and a position offset of the coordinate position, the stabilized video frame is a video frame after the second video frame is compensated for the position offset and the rotation angle.

[0125] In this embodiment, based on the second position offset, the second video frame is compensated for the position offset, and based on the second rotation angle, the second video frame is compensated for the rotation angle, so as to solve the problems of image disorder and poor image quality in the second video frame and improve the image quality in the second video frame.

[0126] In one embodiment, before step S101, that is, before obtaining the first feature point information of the preset physical mark of the first video frame, the inter-frame video stabilization method further includes:

[0127] S1011: Acquire an original video, perform filtering and noise reduction processing on the original video, and acquire a first video to be stabilized;

[0128] S1012: grayscale the first video to be stabilized to obtain a second video to be stabilized;

[0129] S1013: Determine, from the second video to be stabilized, a first video frame and a second video frame that are shot successively.

[0130] The original video refers to the unprocessed video to be stabilized collected in the automobile collision test. The filtering and noise reduction processing refers to the process of removing image noise from the original video. The first video to be stabilized refers to the video obtained after the filtering and noise reduction processing is performed on the original video.

[0131] As an example, in step S1011, the computer device obtains the original video, performs filtering and noise reduction processing on the original video, and obtains the first video to be stabilized after filtering and noise reduction processing. In this example, filtering and noise reduction processing is performed on the original video to facilitate the subsequent more accurate acquisition of the first feature point information containing the preset physical mark.

[0132] The grayscale processing refers to the process of grayscale processing the first video to be stabilized. The second video to be stabilized refers to the video obtained after grayscale processing is performed on the first video to be stabilized.

[0133] As an example, in step S1012, the computer device obtains a first video to be stabilized, performs grayscale processing on the first video to be stabilized, and obtains a second video to be stabilized after grayscale processing. In this example, grayscale processing is performed on the first video to be stabilized to facilitate further accurate acquisition of first feature point information containing a preset physical marker.

[0134] As an example, in step S1013, the computer device determines the first video frame and the second video frame shot successively from the second video to be stabilized after the grayscale processing. In this example, the computer device can start from the first video frame of the second video to be stabilized, take the first video frame of the second video to be stabilized as the first video frame, take the second video frame as the second video frame, and execute steps S101 to S107, perform feature matching on the first two video frames, and perform inverse motion compensation on the second video frame to obtain the corresponding stabilized video frame; take the obtained stabilized video frame as the first video frame, take the third video frame as the second video frame, continue to execute steps S102 to S107, and so on, so as to facilitate the subsequent stabilization processing of each video frame in the second video to be stabilized.

[0135] In this embodiment, the original video is filtered, denoised and grayed to obtain a first video frame and a second video frame shot successively, so that the first feature point information of the preset physical marker of the first video frame can be more accurately obtained and feature matching can be performed.

[0136] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0137] In one embodiment, an inter-frame video stabilization device is provided, and the inter-frame video stabilization device corresponds one-to-one to the inter-frame video stabilization method in the above embodiment. Figure 4 As shown, the inter-frame video stabilization device includes a first feature point information acquisition module 401, a second feature point information acquisition module 402, a first motion offset determination module 403, a target area determination module 404, a third feature point information determination module 405, a second motion offset determination module 406 and a first stabilized video frame acquisition module 407. Each functional module is described in detail as follows:

[0138] A first feature point information acquisition module 401 is used to acquire first feature point information of a preset physical marker of a first video frame, where the first feature point information includes coordinate information and pixel information of the preset physical marker;

[0139] The second feature point information acquisition module 402 performs feature matching on the second video frame based on the pixel information in the first feature point information to acquire the second feature point information of the corresponding preset physical marker in the second video frame, where the second feature point information includes the coordinate information of the preset physical marker;

[0140] A first motion offset determination module 403 is used to determine a first motion offset of the first video frame and the second video frame according to the coordinate information of the first feature point information and the coordinate information of the second feature point information;

[0141] A target region determination module 404 is configured to determine, in the second video frame, a target region including a preset physical marker corresponding to the first video frame if the first motion offset is greater than a preset offset;

[0142] A third feature point information acquisition module 405 is used to acquire third feature point information of a preset physical marker in the target area, where the third feature point information includes coordinate information of the preset physical marker;

[0143] A second motion offset determination module 406, configured to determine a second motion offset of the first video frame and the second video frame according to the coordinate information of the first feature point information and the coordinate information of the third feature point information;

[0144] The first image-stabilized video frame acquisition module 407 performs inverse motion compensation on the second video frame based on the second motion offset to acquire an image-stabilized video frame corresponding to the second video frame.

[0145] Among them, the number of preset physical identifiers is two.

[0146] In another embodiment, the inter-frame video stabilization device further includes:

[0147] The second stabilized video frame acquisition module is used to perform inverse motion compensation on the second video frame based on the first motion offset to acquire a stabilized video frame corresponding to the second video frame if the first motion offset is not greater than a preset offset.

[0148] In another embodiment, the inter-frame video stabilization device further includes:

[0149] A first peak signal-to-noise ratio value determination module is configured to calculate the peak signal-to-noise ratio based on the local area including the preset physical mark in the first video frame and the local area including the preset physical mark in the second video frame to determine a first peak signal-to-noise ratio value;

[0150] A second peak signal-to-noise ratio value determination module is configured to calculate the peak signal-to-noise ratio based on the local area including the preset physical mark in the stabilized video frame corresponding to the first video frame and the local area including the preset physical mark in the stabilized video frame corresponding to the second video frame, so as to determine the second peak signal-to-noise ratio value;

[0151] The image stabilization effect evaluation module determines an image stabilization effect evaluation of the image stabilized video frame based on the first peak signal-to-noise ratio value and the second peak signal-to-noise ratio value.

[0152] In one embodiment, the target area determination module 404 includes:

[0153] The target area acquisition submodule is used to determine a target area containing a preset physical mark of the first video frame in the second video frame if the first position offset is greater than the position offset threshold, or the first rotation angle is greater than the rotation angle threshold.

[0154] Wherein, the first motion offset includes a first position offset and a first rotation angle;

[0155] In one embodiment, the first stabilized video frame acquisition module 107 includes:

[0156] A position offset compensation submodule, which performs position offset compensation on the second video frame based on the second position offset;

[0157] A rotation angle compensation submodule, performing rotation angle compensation on the second video frame based on the second rotation angle;

[0158] The stabilized video frame is a video frame obtained by performing position offset compensation and / or rotation angle compensation on the second video frame. The second motion offset includes a second position offset and a second rotation angle.

[0159] For the specific definition of the inter-frame video stabilization device, please refer to the definition of the inter-frame video stabilization method above, which will not be repeated here. Each module in the above-mentioned inter-frame video stabilization device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0160] In one embodiment, a computer device is provided, whose internal structure diagram can be as follows: Figure 5As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data used or generated during the execution of the inter-frame video stabilization method. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an inter-frame video stabilization method is implemented.

[0161] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the inter-frame video stabilization method in the above embodiment is implemented, for example Figure 1 S101-S107 as shown, or Figure 2 and Figure 3 Alternatively, when the processor executes the computer program, the functions of each module / unit in the embodiment of the inter-frame video stabilization device are realized, for example Figure 4 The functions of the first feature point information acquisition module 401, the second feature point information acquisition module 402, the first motion offset determination module 403, the target area determination module 404, the third feature point information determination module 405, the second motion offset determination module 406 and the first stabilized video frame acquisition module 407 are not repeated here to avoid repetition.

[0162] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the inter-frame video stabilization method in the above embodiment is implemented, for example Figure 1 S101-S107 as shown, or Figure 2 and Figure 3 Alternatively, when the processor executes the computer program, the functions of each module / unit in the embodiment of the inter-frame video stabilization device are realized, for example Figure 4 The functions of the first feature point information acquisition module 401, the second feature point information acquisition module 402, the first motion offset determination module 403, the target area determination module 404, the third feature point information determination module 405, the second motion offset determination module 406 and the first stabilized video frame acquisition module 407 are not described here to avoid repetition. The computer-readable storage medium may be non-volatile or volatile.

[0163] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0164] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. An inter-frame video stabilization method, characterized in that: include: Acquire first feature point information of a preset physical marker of a first video frame, wherein the first feature point information includes coordinate information and pixel information of the preset physical marker; Based on the pixel information in the first feature point information, feature matching is performed on the second video frame to obtain second feature point information of a corresponding preset physical marker in the second video frame, where the second feature point information includes coordinate information of the preset physical marker; Determine a first motion offset of the first video frame and the second video frame according to the coordinate information of the first feature point information and the coordinate information of the second feature point information; If the first motion offset is greater than a preset offset, determining a target area including a preset physical marker corresponding to the first video frame in the second video frame; Acquire third feature point information of a preset physical marker in the target area, wherein the third feature point information includes coordinate information of the preset physical marker; Determine a second motion offset of the first video frame and the second video frame according to the coordinate information of the first feature point information and the coordinate information of the third feature point information; Based on the second motion offset, inverse motion compensation is performed on the second video frame to obtain a stabilized video frame corresponding to the second video frame.

2. The inter-frame video stabilization method according to claim 1, wherein: The number of the preset physical identifiers is two.

3. The inter-frame video stabilization method according to claim 1, wherein: After determining the first motion offset of the first video frame and the second video frame according to the coordinate information of the first feature point information and the coordinate information of the second feature point information, the inter-frame video stabilization method further includes: If the first motion offset is not greater than a preset offset, inverse motion compensation is performed on the second video frame based on the first motion offset to obtain a stabilized video frame corresponding to the second video frame.

4. The inter-frame video stabilization method according to claim 1, wherein: After acquiring the stabilized video frame corresponding to the second video frame, the inter-frame video stabilization method further includes: Performing peak signal-to-noise ratio calculation based on the local area including the preset physical marker in the first video frame and the local area including the preset physical marker in the second video frame to determine a first peak signal-to-noise ratio value; Determine a second peak signal-to-noise ratio value by performing peak signal-to-noise ratio calculation based on a local area including the preset physical marker in the stabilized video frame corresponding to the first video frame and a local area including the preset physical marker in the stabilized video frame corresponding to the second video frame; An image stabilization effect evaluation of the image stabilized video frame is determined based on the first peak signal-to-noise ratio value and the second peak signal-to-noise ratio value.

5. The inter-frame video stabilization method according to claim 1, wherein: The first motion offset includes a first position offset and a first rotation angle; If the first motion offset is greater than a preset offset, determining a target area including a preset physical mark corresponding to the first video frame in the second video frame includes: If the first position offset is greater than a position offset threshold, or the first rotation angle is greater than a rotation angle threshold, a target area including a preset physical mark of the first video frame is determined in the second video frame.

6. The inter-frame video stabilization method according to claim 1, wherein: The second motion offset includes a second position offset and a second rotation angle; The performing inverse motion compensation on the second video frame based on the second motion offset to obtain a stabilized video frame corresponding to the second video frame includes: Based on the second position offset, performing position offset compensation on the second video frame; Based on the second rotation angle, performing rotation angle compensation on the second video frame; The stabilized video frame is a video frame obtained by performing position offset compensation and / or rotation angle compensation on the second video frame.

7. An inter-frame video stabilization device, characterized in that: include: A first feature point information acquisition module, used to acquire first feature point information of a preset physical marker of a first video frame, wherein the first feature point information includes coordinate information and pixel information of the preset physical marker; A second feature point information acquisition module, based on the pixel information in the first feature point information, performs feature matching on the second video frame to acquire second feature point information of a corresponding preset physical marker in the second video frame, wherein the second feature point information includes coordinate information of the preset physical marker; A first motion offset determination module, used to determine a first motion offset of the first video frame and the second video frame according to the coordinate information of the first feature point information and the coordinate information of the second feature point information; a target region determining module, configured to determine, in the second video frame, a target region including a preset physical marker corresponding to the first video frame if the first motion offset is greater than a preset offset; A third feature point information acquisition module, used to acquire third feature point information of a preset physical marker in the target area, wherein the third feature point information includes coordinate information of the preset physical marker; A second motion offset determination module, used to determine a second motion offset of the first video frame and the second video frame according to the coordinate information of the first feature point information and the coordinate information of the third feature point information; The first stabilized video frame acquisition module performs inverse motion compensation on the second video frame based on the second motion offset to acquire a stabilized video frame corresponding to the second video frame.

8. The inter-frame video stabilization device according to claim 7, wherein: The inter-frame video stabilization device also includes: A first peak signal-to-noise ratio value determination module, which performs peak signal-to-noise ratio calculation based on a local area including a preset physical marker in the first video frame and a local area including the preset physical marker in the second video frame to determine a first peak signal-to-noise ratio value; a second peak signal-to-noise ratio value determining module, performing peak signal-to-noise ratio calculation based on a local area including the preset physical marker in the stabilized video frame corresponding to the first video frame and a local area including the preset physical marker in the stabilized video frame corresponding to the second video frame, to determine a second peak signal-to-noise ratio value; The image stabilization effect evaluation module determines an image stabilization effect evaluation of the image stabilized video frame based on the first peak signal-to-noise ratio value and the second peak signal-to-noise ratio value.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the inter-frame video stabilization method according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the inter-frame video stabilization method according to any one of claims 1 to 6 is implemented.

Images