Video processing method and device, equipment, storage medium and computer program product

By obtaining the moving layer parameter information in the video for video frame cropping, the problem of poor mirroring effect in the prior art is solved, and personalized highlighting of the picture content and improving the video display effect are achieved.

CN119052394APending Publication Date: 2024-11-29MIGU DIGITAL MEDIA CO LTD +2
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411083355.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-08
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

In the prior art, the video processing solution has poor mirroring effect, and cannot achieve personalized and outstanding screen content, resulting in poor video display effect.

Method used

By obtaining the motion parameter information of the moving layer in the target video, calculating the picture cropping coefficient of the video frame, determining the size and position of the cropping area, performing video cropping, realizing the pseudo-moving mirror effect, and enhancing the personalized expression of the picture content.

Benefits of technology

It effectively improves the video display effect, solves the problem of poor mirroring effect, achieves personalized highlighting of the picture content, and enhances the visual experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119052394A_ABST
    Figure CN119052394A_ABST
Patent Text Reader

Abstract

The invention provides a video processing method and device, equipment, a storage medium and a computer program product, and the method comprises the steps: obtaining the motion parameter information of a motion layer in a target video; acquiring a picture cutting coefficient of a video frame of the target video according to the motion parameter information; the picture clipping coefficient is used for determining the size of a clipping area of a video frame; and cutting the target video according to the picture cutting coefficient. According to the scheme, video cutting according to the video motion layer can be supported, and a pseudo-motion mirror (for example, exaggerated display is performed on a motion object, for example, the twisting span amplitude of a dancer is increased through cutting) is realized, so that the purpose of individualized highlighting of picture content is achieved, the final display effect of the video is improved, and the user experience is improved. The problem that the mirror moving effect of a video processing scheme in the prior art is poor is well solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of video technology, and in particular to a video processing method, apparatus, device, storage medium and computer program product. Background Art

[0002] The existing technology recognizes and tracks the preset model, which requires the target scene or other preset environmental parameters as input, and recognizes, processes, and trains the images collected by the specific preset parameters to obtain the corresponding camera movement rules or methods, and uses the rules and methods to adjust the shooting angle in real time or track the target area. However, this camera movement method cannot achieve personalized highlighting of the image content, resulting in poor final video display effect.

[0003] As described above, the video processing solutions in the prior art have problems such as poor camera movement effect. Summary of the invention

[0004] The purpose of the present application is to provide a video processing method, apparatus, device, storage medium and computer program product to solve the problem of poor camera movement effect of video processing solutions in the prior art.

[0005] In order to solve the above technical problems, the embodiment of the present application provides a video processing method, including:

[0006] Obtain motion parameter information of the motion layer in the target video;

[0007] According to the motion parameter information, a screen cropping coefficient of the video frame of the target video is obtained; the screen cropping coefficient is used to determine the size of the cropping area of ​​the video frame;

[0008] The target video is cropped according to the picture cropping coefficient.

[0009] Optionally, the target video includes a first video frame and a second video frame that are adjacent to each other;

[0010] The step of obtaining motion parameter information of a motion layer in a target video includes:

[0011] Obtaining a first center point position of the motion layer in the first video frame and a second center point position of the motion layer in the second video frame;

[0012] Obtaining displacement information of the motion layer according to the first center point position and the second center point position;

[0013] According to the displacement information, motion parameter information of the motion layer in the target video is obtained.

[0014] Optionally, acquiring a screen cropping coefficient of a video frame of the target video according to the motion parameter information includes:

[0015] The image cropping coefficient of the video frame is obtained according to the cropping amplitude and the motion parameter information.

[0016] Optionally, the motion parameter information includes: acceleration information and displacement information; the acceleration information includes: a first acceleration corresponding to a first coordinate axis in a reference coordinate system, and a second acceleration corresponding to a second coordinate axis in the reference coordinate system; the displacement information is the displacement of the motion layer in the reference coordinate system within a first time length;

[0017] The picture cropping coefficient includes: a first cropping coefficient corresponding to a first coordinate axis in the reference coordinate system, and a second cropping coefficient corresponding to a second coordinate axis in the reference coordinate system;

[0018] The acquiring the picture cropping coefficient of the video frame according to the cropping amplitude and the motion parameter information includes:

[0019] Obtaining, according to the displacement information, a first motion-related influence factor corresponding to the first coordinate axis and a second motion-related influence factor corresponding to the second coordinate axis;

[0020] Obtaining the first cropping coefficient according to the cropping amplitude, the first acceleration and the first motion-related influencing factor;

[0021] The second cropping coefficient is obtained according to the cropping amplitude, the second acceleration and the second motion-related influencing factor.

[0022] Optionally, cropping the target video according to the picture cropping coefficient includes:

[0023] Determining the size of the cropping area of ​​each video frame of the target video according to the picture cropping coefficient;

[0024] Determining the position of the cropping area of ​​each video frame according to the motion direction information of the motion layer;

[0025] The corresponding video frame is cropped according to the size and position.

[0026] Optionally, also include:

[0027] Identify a motion layer in the target video, and use a layer portion other than the motion layer as a background layer;

[0028] The origin of the reference coordinate system is determined in the background layer.

[0029] The embodiment of the present application also provides a video processing device, including:

[0030] A first acquisition module is used to acquire motion parameter information of a motion layer in a target video;

[0031] A second acquisition module is used to acquire a picture cropping coefficient of a video frame of the target video according to the motion parameter information; the picture cropping coefficient is used to determine the size of a cropping area of ​​the video frame;

[0032] The first processing module is used to crop the target video according to the picture cropping coefficient.

[0033] Optionally, the target video includes a first video frame and a second video frame that are adjacent to each other;

[0034] The step of obtaining motion parameter information of a motion layer in a target video includes:

[0035] Obtaining a first center point position of the motion layer in the first video frame and a second center point position of the motion layer in the second video frame;

[0036] Obtaining displacement information of the motion layer according to the first center point position and the second center point position;

[0037] According to the displacement information, motion parameter information of the motion layer in the target video is obtained.

[0038] Optionally, acquiring a screen cropping coefficient of a video frame of the target video according to the motion parameter information includes:

[0039] The image cropping coefficient of the video frame is obtained according to the cropping amplitude and the motion parameter information.

[0040] Optionally, the motion parameter information includes: acceleration information and displacement information; the acceleration information includes: a first acceleration corresponding to a first coordinate axis in a reference coordinate system, and a second acceleration corresponding to a second coordinate axis in the reference coordinate system; the displacement information is the displacement of the motion layer in the reference coordinate system within a first time length;

[0041] The picture cropping coefficient includes: a first cropping coefficient corresponding to a first coordinate axis in the reference coordinate system, and a second cropping coefficient corresponding to a second coordinate axis in the reference coordinate system;

[0042] The acquiring the picture cropping coefficient of the video frame according to the cropping amplitude and the motion parameter information includes:

[0043] Obtaining, according to the displacement information, a first motion-related influence factor corresponding to the first coordinate axis and a second motion-related influence factor corresponding to the second coordinate axis;

[0044] Obtaining the first cropping coefficient according to the cropping amplitude, the first acceleration and the first motion-related influencing factor;

[0045] The second cropping coefficient is obtained according to the cropping amplitude, the second acceleration and the second motion-related influencing factor.

[0046] Optionally, cropping the target video according to the picture cropping coefficient includes:

[0047] Determining the size of the cropping area of ​​each video frame of the target video according to the picture cropping coefficient;

[0048] Determining the position of the cropping area of ​​each video frame according to the motion direction information of the motion layer;

[0049] The corresponding video frame is cropped according to the size and position.

[0050] Optionally, also include:

[0051] A second processing module is used to identify a motion layer in the target video and use a layer portion other than the motion layer as a background layer;

[0052] The first determination module is used to determine the origin of the reference coordinate system in the background layer.

[0053] An embodiment of the present application also provides a video processing device, including a memory, a processor, and a program stored in the memory and executable on the processor; when the processor executes the program, the above-mentioned video processing method is implemented.

[0054] The embodiment of the present application also provides a readable storage medium on which a program is stored. When the program is executed by a processor, the steps in the above-mentioned video processing method are implemented.

[0055] An embodiment of the present application further provides a computer program product, including computer instructions, which implement the steps of the above-mentioned video processing method when executed by a processor.

[0056] The beneficial effects of the above technical solution of the present application are as follows:

[0057] In the above scheme, the video processing method obtains the motion parameter information of the motion layer in the target video; obtains the screen cropping coefficient of the video frame of the target video according to the motion parameter information; the screen cropping coefficient is used to determine the size of the cropping area of ​​the video frame; and the target video is cropped according to the screen cropping coefficient. It can support video cropping according to the video motion layer to achieve pseudo-camera movement (such as exaggerating the display of the moving object, for example, increasing the amplitude of the dancer's twisting through cropping, etc.), so as to achieve the purpose of highlighting the personalized content of the picture, improve the final display effect of the video, and well solve the problem of poor camera movement effect of the video processing scheme in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A schematic diagram of a video processing method flow in an embodiment of the present application;

[0059] Figure 2 A schematic diagram of a specific implementation flow of the video processing method according to an embodiment of the present application;

[0060] Figure 3 The motion target of the embodiment of the present application is shown as follows Figure 1 ;

[0061] Figure 4 The motion target of the embodiment of the present application is shown as follows Figure 2 ;

[0062] Figure 5 The motion layer diagram of the embodiment of the present application is shown in FIG. Figure 1 ;

[0063] Figure 6 A schematic diagram of background layer identification according to an embodiment of the present application;

[0064] Figure 7 A schematic diagram of the center coordinates of a motion layer in an embodiment of the present application;

[0065] Figure 8 A schematic diagram of virtual coordinates of an embodiment of the present application;

[0066] Fig. 9 The motion layer diagram of the embodiment of the present application is shown in FIG. Figure 2 ;

[0067] Fig.10 A schematic diagram of a motion layer displacement interval in an embodiment of the present application;

[0068] Fig.11 A schematic diagram of camera movement according to an embodiment of the present application;

[0069] Fig.12 This is a schematic diagram of the screen cropping of the embodiment of the present application. Figure 1 ;

[0070] Fig.13 This is a schematic diagram of the screen cropping of the embodiment of the present application. Figure 2 ;

[0071] Fig.14 Schematic diagram of the structure of a video processing device according to an embodiment of the present application. DETAILED DESCRIPTION

[0072] In order to make the technical problems, technical solutions and advantages to be solved by the present application clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0073] This application aims to solve the problem that the camera movement effect of the video processing solution in the existing technology is poor, and provides a video processing method, such as Figure 1 As shown, including:

[0074] Step 11: Obtain motion parameter information of the motion layer in the target video.

[0075] Among them, the target video is a continuous video, for example: there is no transition in the target video (for example, the degree of pixel change per unit time is less than a threshold); if a long video is switched between shots, then the continuous segments before each shot or between shots can be used as a target video.

[0076] Specific examples include Figure 2 As shown, this program also includes:

[0077] 1. Obtain continuous target video;

[0078] For example, in a long video, there are multiple transitions between scenes. The target video obtained here is the video clip without transitions in the long video. One long video can obtain multiple target videos. For example, if a dancer dances passionately, if the dancer is continuously filmed in one shot and fixed position, then the long video is a continuous target video; if the long video switches the dancer's scenes, then the continuous clips before each scene or between scenes are a target video.

[0079] 2. Motion layer recognition;

[0080] The position change of the moving target in the target video (the change in the visual canvas per unit time) is the strongest. For example: Figure 3 In the middle, the dancer’s limbs change frequently throughout the video and can be identified as motion layers; Figure 4 In the example, the hips of the dancer have strong displacement, and the hips are identified as a motion layer. The motion layer can be determined based on information such as the speed of change and the area, but is not limited thereto.

[0081] Specifically, the video screen can be divided into pixel grids, and the motion layer is a continuous pixel grid unit. There will only be one motion layer in the canvas displayed by the video screen, and it is almost unchanged in the entire target video. In other words, the motion layer is the part that changes most drastically in the pixel grid matrix. The motion layer itself does not have a specific size, but the smaller the motion layer is accurately identified, the better the target effect produced by the subsequent cropping steps. In this solution, you can set the continuous part of the first 20% of the pixels in the screen that change most drastically as the motion layer. For example, Figure 5 The video shown shows the irregular pattern of the motion layer A identified at time T.

[0082] Step 12: According to the motion parameter information, a screen cropping coefficient of the video frame of the target video is obtained; the screen cropping coefficient is used to determine the size of the cropping area of ​​the video frame.

[0083] The picture cropping coefficient may be a cropping coefficient for one frame, and each video frame corresponds to a picture cropping coefficient (the same or different), but the present invention is not limited thereto.

[0084] Step 13: Crop the target video according to the image cropping coefficient.

[0085] In the present application, after step 13, the following may be included: enlarging the cropped video screen; and displaying the enlarged screen; wherein, "enlarging" may be enlarging according to the size of the visual canvas, such as enlarging to the same size, which is not limited here.

[0086] The video processing method provided in the embodiment of the present application obtains motion parameter information of a motion layer in a target video; obtains a screen cropping coefficient of a video frame of the target video according to the motion parameter information; the screen cropping coefficient is used to determine the size of a cropping area of ​​a video frame; and the target video is cropped according to the screen cropping coefficient. The method can support video cropping according to a video motion layer to achieve pseudo camera movement (such as exaggerating a moving object, for example, increasing the amplitude of a dancer's twisting through cropping, etc.), thereby achieving the purpose of highlighting the personalized content of the picture, improving the final display effect of the video, and well solving the problem of poor camera movement effect of video processing solutions in the prior art.

[0087] Furthermore, the video processing method also includes: identifying the motion layer in the target video, and using the layer part other than the motion layer as the background layer; determining the origin of the reference coordinate system in the background layer. Based on this, the present scheme can achieve the situation that the displacement interval of the dynamic layer (i.e., the motion layer) in the virtual reference coordinate system established with the background layer remains unchanged through dynamic cropping of the picture, and visually amplifies the displacement interval ratio of the dynamic layer in the cropped picture, thereby finally achieving the lens feeling of "push, pull, shake, shift, follow, throw, lift and lower" of the picture. Specifically, it can be as follows Figure 2 As shown, this solution also includes the following operations:

[0088] 3. Background layer identification;

[0089] All non-moving layers will be identified as background layers, such as Figure 6 As shown, in the video, because the camera follows the athlete and moves horizontally from right to left, the athlete is always in the center of the video, and the athlete is the background layer. The bushes can be identified as the motion layer because they move from left to right relative to the video; corresponding to the above identification of the motion layer in the target video, and the layer part except the motion layer is used as the background layer. The target video in this solution can be a video shot with a still camera or a video shot with a moving camera, which is not limited here.

[0090] 4. Determine the center reference point of the background layer;

[0091] In this scheme, a continuous pixel graphic composed of the pixels with the least drastic changes in the background layer can be used as the origin, that is, the origin of the virtual reference coordinate system (corresponding to the origin of the reference coordinate system determined in the background layer as mentioned above), and any other pixel point in the picture can obtain its coordinates in the virtual coordinate system (that is, the virtual reference coordinate system).

[0092] Among them, the calculation function of the center point of irregular geometric figures can find the center point of the layer. The specific method is as follows:

[0093] Put any irregular shape into a rectangular coordinate system and calculate the coordinates of all pixels in the shape (X1, Y1), ... (Xn, Yn), then the center coordinates are ((X1+... +Xn) / n, (Y1+... Yn) / n);

[0094] like Figure 7 As shown, in Figure 5 The center coordinates of the motion layer at time T can be calculated according to the above formula, which is the coordinates of point B.

[0095] The same calculation method can be used to find the center reference point of the background layer. To simplify the calculation for the sake of convenience, we assume that Figure 8The video angle and shooting point in the video do not shift at all, because the background wall in the video does not shift relative to the shooting point. The virtual coordinate center reference point can be obtained, which is the center point of the entire rectangular screen, such as Figure 8 Point C in .

[0096] In the embodiment of the present application, the target video includes a first adjacent video frame and a second adjacent video frame; the step of obtaining the motion parameter information of the motion layer in the target video includes: obtaining the first center point position of the motion layer in the first video frame and the second center point position of the motion layer in the second video frame; obtaining the displacement information of the motion layer based on the first center point position and the second center point position; obtaining the motion parameter information of the motion layer in the target video based on the displacement information. In this way, the motion parameter information of the motion layer can be accurately obtained. Specifically, Figure 2 As shown, this solution may also include the following operations:

[0097] 5. Get the displacement interval of the motion layer in the visible canvas (which can also be understood as the displacement interval in the picture);

[0098] The displacement interval of the motion layer can be the displacement interval of the change of the position of the center point of the motion layer, such as Fig. 9 As shown, assuming that the X-axis and Y-axis of the canvas size are exactly (1000,1000), assuming that the vertex coordinates of the interval of the motion layer are (300,400), (500,400), (300,300), (500,300), then the center coordinates of the motion layer are (400,350). Assuming that the dancer dances continuously, Fig.10 As shown in the figure, the vertex coordinates of the interval to which the motion layer moves in time T are (450,420), (650,420), (450,320), (650,320), and the center coordinates of the motion layer move to (550,370) in time T. Based on this, it can be concluded that the motion displacement of the motion layer in the virtual coordinate system in time T is (150,20).

[0099] 6. (Optional) Get the displacement interval of the motion layer relative to the background layer reference point;

[0100] The same as operation 5, because the assumption is made that the video shooting source is fixed, the displacement interval in the virtual reference coordinate system (here it is assumed that the origin of the coordinate system is aligned with the lower left corner of the canvas) can also be obtained as (150, 20). In this application, the displacement intervals of the motion layer obtained in operations 5 and 6 can be used one by one, or the displacement interval obtained in operation 6 can be used after adjusting the displacement interval obtained in operation 5, or the displacement interval obtained in operation 5 can be used directly, etc., which is not limited here.

[0101] In the embodiment of the present application, the obtaining of the screen cropping coefficient of the video frame of the target video according to the motion parameter information includes: obtaining the screen cropping coefficient of the video frame according to the cropping amplitude and the motion parameter information. The cropping amplitude can be set according to user needs, so as to better meet the needs of different users. Specifically, the motion parameter information may include: acceleration information and displacement information; the acceleration information may include: a first acceleration corresponding to a first coordinate axis in a reference coordinate system, and a second acceleration corresponding to a second coordinate axis in the reference coordinate system; the displacement information may be the displacement of the motion layer in the reference coordinate system within a first duration; the screen cropping coefficient includes: a first cropping coefficient corresponding to a first coordinate axis in the reference coordinate system, and a second cropping coefficient corresponding to a second coordinate axis in the reference coordinate system; the step of obtaining the screen cropping coefficient of the video frame according to the cropping amplitude and the motion parameter information includes: obtaining a first motion-related influence factor corresponding to the first coordinate axis and a second motion-related influence factor corresponding to the second coordinate axis according to the displacement information; obtaining the first cropping coefficient according to the cropping amplitude, the first acceleration and the first motion-related influence factor; obtaining the second cropping coefficient according to the cropping amplitude, the second acceleration and the second motion-related influence factor. In this way, the screen cropping coefficient can be obtained specifically, wherein the first acceleration and the second acceleration may be the same or different, which is not limited here.

[0102] The motion parameter information may also include: at least one of motion direction information and speed information; and / or, the reference coordinate system may be any coordinate system capable of determining the motion parameter information; and / or, the first duration may include: the interval duration between adjacent first video frames and second video frames; the displacement includes the displacement between the position in the first video frame and the position in the second video frame; and / or, the screen cropping coefficient is a cropping coefficient for the second video frame (the first frame of the video may be cropped according to default parameters), but is not limited thereto. Specifically, it may be as follows Figure 2 As shown, this solution also includes the following operations:

[0103] 7. Obtain the clipping coefficient according to the displacement interval ratio;

[0104] Assuming that this solution is not implemented, in order to achieve the same effect, Fig. 9 When shooting the hip twisting action, the camera needs to move along with the hip movement direction or the shooting angle needs to be tilted to the right, or a combination of the two. Fig.11 As shown (the camera moves from S1 to S2): when the motion layer moves to the right, the camera needs to use professional camera techniques such as relative displacement or relative tilt to achieve the effect of moving the motion point from relatively left to relatively right in the camera's image, thereby amplifying the motion amplitude of the motion point in the image captured by the camera.

[0105] The current operation of this solution is different from the existing technical solution. The camera does not need to perform professional lens movement changes. It only needs to be still. In the scene of twisting the hips, the hips will be identified as the moving layer. Assuming that the camera moves relative to the hips, the hips will be identified as the background layer. To facilitate the explanation of this solution, continue to analyze under the condition of still lens:

[0106] Under the premise that the lens is stationary, that is, continue to follow the description of operations 4 to 6, the center point of the motion layer at time T1 is (400,350), and the center point of the motion layer at time T2 is (550,370). During the △T time of T2-T1, the displacement range of the motion layer relative to the canvas where the original image H0 is collected without cropping is: 150 on the X axis and 20 on the Y axis, that is, the displacement ratio of the hips relative to the image during the △T time is (15%, 2%); now the original image is cropped according to a certain algorithm, and the following factors affect the cropping coefficient:

[0107] ① The acceleration of the motion (corresponding to the above acceleration information): g. For example, at time t, the faster the acceleration of the center of the motion layer, the faster the image is cropped.

[0108] ② The x-axis and y-axis displacements of the △T time (corresponding to the above displacement information): △x, △y. For example, the larger the range of movement of the moving layer on the x-axis and y-axis during the △T time, the larger the area cropped from the original image.

[0109] ③ Preset cropping amplitude: c. For example, c is a constant, which can be preset (for example, according to experience information or user demand information) to achieve an amplitude ratio value for amplifying the motion effect.

[0110] ④ x-axis and y-axis motion correlation influence factors: px (corresponding to the first motion correlation influence factor mentioned above), py (corresponding to the second motion correlation influence factor mentioned above). The motion of the motion layer between the x-axis and the y-axis will affect each other, and the realism and immersion of the lens movement effect can be improved according to this factor. It can also be understood that the movement between the x-axis and the y-axis is set to affect each other, which increases the connection between the two directional movements and further improves the realism of the video movement.

[0111] The calculation of py can be:

[0112] The calculation of px can be:

[0113] The motion acceleration g within the △T time can be obtained by the current method, but is not limited to this;

[0114] Based on the above factors, the cropping coefficients for the image on the x-axis and y-axis are:

[0115] Corresponding to the above, according to the clipping amplitude, the first acceleration and the first motion-related influencing factor, the first clipping coefficient is obtained;

[0116] Corresponding to the above, the second cropping coefficient is obtained according to the cropping amplitude, the second acceleration and the second motion-related influencing factor.

[0117] It is explained here that the above cropping coefficients are calculated based on the cropping amplitude and motion parameters, so that the image cropping amount is associated with the motion parameters, which can enhance the realism of the video camera movement.

[0118] In addition, consider that: because the cropping of the picture does not affect the size of the visible canvas, it only enlarges the relative proportion of the picture (the resolution of the picture can be reduced). To simplify the calculation, assume that the cropping size on the X axis and Y axis within the △T time is 200 and 20 respectively (this can be the initial setting size), such as Fig.12 The coordinate ratio of the center points R1 and R2 of the motion layer at the time T1 and T2 in the cropped images H1 and H2 is calculated as shown. The cropped image can be stretched to the size of the original image, and the coordinates of the center point after cropping may not be directly equal to the original coordinates minus the cropping amount; however, for the convenience of description and to highlight the effect, the coordinates before stretching (the position of the cropped image before stretching) will be used as an example for numerical calculation, that is, standing in the original reference coordinate system to give an example: the center coordinates after cropping = the original coordinates - the cropping amount; the details are as follows:

[0119] At time T1, the coordinates of R1 in the uncropped image H0 are (400, 350), and the coordinates of R1 in the cropped image H1 are (200, 330). The coordinate ratio of R1 in the H0 image is (40%, 35%), and the coordinate ratio in the H1 image is (20%, 33.67%).

[0120] At time T2, the coordinates of R2 in the uncropped image H0 are (550, 370), and the coordinates of R2 in the cropped image H2 are (550, 370). The coordinate ratio of R2 in the H0 image is (55%, 37%), and the coordinate ratio in the H2 image is (68.75%, 37.75%).

[0121] As mentioned above, if the displacement of the motion layer in the visual canvas of the uncropped picture moves from (40%, 35%) to (55%, 37%), the displacement ratio is (15%, 2%); while the displacement of the cropped picture in the visual canvas moves from (20%, 33.67%) to (68.75%, 37.75%), the displacement ratio is (48.75%, 4.8%). The displacement range of the cropped picture is 3.25 times and 2.4 times on the X-axis and Y-axis respectively compared to the uncropped picture, which creates a certain visual magnification in terms of visual effect. It should be noted that in this solution, the lower left corner of the picture before and after cropping can be aligned with the origin of the coordinate system, but this is not limited to this.

[0122] In an embodiment of the present application, the cropping of the target video according to the picture cropping coefficient includes: determining the size of the cropping area of ​​each video frame of the target video according to the picture cropping coefficient; determining the position of the cropping area of ​​each video frame according to the motion direction information of the motion layer (for example, if the dancer twists to the left, the cropping position on the left side can be selected, etc.); and cropping the corresponding video frame according to the size and position. In this way, accurate video cropping can be performed. Determining the size of the cropping area of ​​each video frame of the target video according to the picture cropping coefficient includes: determining the first size of the cropping area of ​​the video frame extending along the first coordinate axis according to the first cropping coefficient; determining the second size of the cropping area of ​​the video frame extending along the second coordinate axis according to the second cropping coefficient. Specifically, it can be as follows Figure 2 As shown, this solution also includes the following operations:

[0123] 8. Dynamic cropping of images;

[0124] Continue to follow Figure 2 For example, the contents of operations 4 to 7 in Fig.13 As shown in FIG. 1 , the image cropping action within the △T time is shown. At time T1, the C1 part (cropping area) is completely cropped, as shown in FIG. Fig.13 As shown on the left, at time T1, the coordinates of the motion layer in the uncropped image are (400,350), and the coordinates in the cropped image are (200,330). This makes the visual center of gravity of the motion layer at time T1 further to the left than that of the uncropped image.

[0125] When the movement reaches time T2, the C2 part (cropped area) has been completely cropped (smooth cropping), and the C1 part has been completely restored visually (smooth restoration). Fig.13As shown on the right, at time T2, the coordinates of the motion layer in the uncropped image are (550,370), and the coordinates in the cropped image are also (550,370), but the X axis of the image is cropped from 1000 to 800. This makes the visual center of gravity of the motion layer further to the right than the uncropped image. Based on this, the motion amplitude of the motion layer in the visible canvas (the canvas used to display the image) during the △T time is visually enhanced compared to when the image is not cropped.

[0126] In this solution, the clipping coefficient can be used to determine the size of the clipping area (such as f(x) is used to determine Fig.13 The width D of the left part of the cropped area in the left figure (i.e., the horizontal coordinate size of the cropped part that is the same as the vertical size of the picture); f(y) is used to determine Fig.13 The height E of the right part of the cropping area in the left figure (i.e., the vertical coordinate size of the cropping part that is the same as the horizontal size of the picture); the cropping area can include 4 shapes (see Fig.13 There are two types shown: └ and ┐, and can also include: ┌ and ┘); wherein the cropping area of ​​each shape can be aligned with the edge of the picture, such as Fig.13 As shown in the figure, which shape to use can be determined according to the movement target in the figure. For example, if the dancer twists his hips to the left, Fig.13 The lower left area (i.e., shape └) shown on the left can be cropped, or the upper left area (i.e., shape ┌) can be cropped; further, the shape └ or shape ┌ can be determined based on the position of the motion layer in the picture. For example, if the motion layer is close to the upper side, the upper left area (i.e., shape ┌) can be cropped, which can further highlight the motion amplitude, or the lower left area (i.e., shape └) can be cropped, which can ensure that the motion layer is in the middle of the picture as much as possible to meet the personal viewing needs of some users, which is not limited here. This part of the operation can correspond to the above-mentioned determination of the size of the cropping area of ​​each video frame of the target video according to the picture cropping coefficient; determination of the position of the cropping area of ​​each video frame according to the motion direction information of the motion layer; and cropping of the corresponding video frame according to the size and position.

[0127] 9. Get the final video;

[0128] As shown in operation 8, the final cropped video can be obtained after smooth cropping.

[0129] Based on the above, the solution provided by the embodiment of the present application can be specifically implemented as a method for generating video camera movement clips through motion behavior, which can identify motion layers and the displacement of the center point of the motion layers, and then crop the picture accordingly to achieve pseudo camera movement. This processing can not only simulate the illusion of adjusting the lens according to the dynamic needs of the scene, but also use photography techniques such as push-pull, pan, follow, and lift to enhance the depth of visual narrative and emotional expression. For example, the video of a cheetah chasing an antelope analyzes and horizontally crops the picture according to the direction of movement, showing the speed and urgency of the hunter chasing the prey; the dancing video sways and swings the video screen according to the extension and combination of the dancer's head, hands, feet and hips to show the dancer's sense of rhythm; the video of a boxing match follows and rotates according to the athlete's footsteps, defensive dodges, and heavy punches, showing the skills and sense of power of the game.

[0130] In summary, this solution can achieve: identifying the motion layer in the picture, and calculating the ratio of the displacement interval of the motion layer in the picture; then dynamically cropping the original picture according to the ratio and the size of the picture, so that the displacement amplitude of the moving target in the picture is visually enhanced compared with the original video on the display device, so as to achieve the purpose of pseudo-camera movement. Specifically, in the case of video camera movement, the picture can be dynamically cropped according to the movement state of the moving target on the user interface.

[0131] This solution does not require any preset system goals, does not require adjustment of the position and angle of the shooting source, and can achieve the visual effect of camera movement by processing the already formed video.

[0132] It is to be noted that this solution can be used to process rhythmic and regular videos such as dances or sports events to obtain a visual effect that is more consistent with the rhythm content of the video, but is not limited to this.

[0133] The present application also provides a video processing device, such as Fig.14 As shown, including:

[0134] The first acquisition module 141 is used to acquire motion parameter information of a motion layer in a target video;

[0135] A second acquisition module 142 is used to acquire a picture cropping coefficient of the video frame of the target video according to the motion parameter information; the picture cropping coefficient is used to determine the size of the cropping area of ​​the video frame;

[0136] The first processing module 143 is configured to crop the target video according to the image cropping coefficient.

[0137] The video processing device provided in the embodiment of the present application obtains motion parameter information of a motion layer in a target video; obtains a screen cropping coefficient of a video frame of the target video according to the motion parameter information; the screen cropping coefficient is used to determine the size of a cropping area of ​​a video frame; and crops the target video according to the screen cropping coefficient. The device can support video cropping according to a video motion layer to achieve pseudo camera movement (such as exaggerating a moving object, for example, increasing the amplitude of a dancer's twisting through cropping), thereby achieving the purpose of highlighting the personalized content of the picture, improving the final display effect of the video, and effectively solving the problem of poor camera movement effect of video processing solutions in the prior art.

[0138] Wherein, the target video includes an adjacent first video frame and a second video frame; the step of obtaining the motion parameter information of the motion layer in the target video includes: obtaining the first center point position of the motion layer in the first video frame and the second center point position of the motion layer in the second video frame; obtaining the displacement information of the motion layer according to the first center point position and the second center point position; and obtaining the motion parameter information of the motion layer in the target video according to the displacement information.

[0139] In the embodiment of the present application, obtaining the screen cropping coefficient of the video frame of the target video according to the motion parameter information includes: obtaining the screen cropping coefficient of the video frame according to the cropping amplitude and the motion parameter information.

[0140] Among them, the motion parameter information includes: acceleration information and displacement information; the acceleration information includes: a first acceleration corresponding to a first coordinate axis in a reference coordinate system, and a second acceleration corresponding to a second coordinate axis in the reference coordinate system; the displacement information is the displacement of the motion layer in the reference coordinate system within a first time length; the screen cropping coefficient includes: a first cropping coefficient corresponding to the first coordinate axis in the reference coordinate system, and a second cropping coefficient corresponding to the second coordinate axis in the reference coordinate system; obtaining the screen cropping coefficient of the video frame according to the cropping amplitude and the motion parameter information includes: obtaining a first motion-related influence factor corresponding to the first coordinate axis and a second motion-related influence factor corresponding to the second coordinate axis according to the displacement information; obtaining the first cropping coefficient according to the cropping amplitude, the first acceleration and the first motion-related influence factor; obtaining the second cropping coefficient according to the cropping amplitude, the second acceleration and the second motion-related influence factor.

[0141] In an embodiment of the present application, cropping the target video according to the picture cropping coefficient includes: determining the size of the cropping area of ​​each video frame of the target video according to the picture cropping coefficient; determining the position of the cropping area of ​​each video frame according to the motion direction information of the motion layer; and cropping the corresponding video frame according to the size and position.

[0142] Furthermore, the video processing device also includes: a second processing module, used to identify the motion layer in the target video and use the layer part other than the motion layer as the background layer; a first determination module, used to determine the origin of the reference coordinate system in the background layer.

[0143] Among them, the implementation embodiments of the above-mentioned video processing method are all applicable to the embodiments of the video processing device and can achieve the same technical effects.

[0144] An embodiment of the present application also provides a video processing device, including a memory, a processor, and a program stored in the memory and executable on the processor; when the processor executes the program, the above-mentioned video processing method is implemented.

[0145] Among them, the implementation embodiments of the above-mentioned video processing method are all applicable to the embodiments of the video processing device and can achieve the same technical effects.

[0146] The embodiment of the present application also provides a readable storage medium on which a program is stored. When the program is executed by a processor, the steps in the above-mentioned video processing method are implemented.

[0147] Among them, the implementation embodiments of the above-mentioned video processing method are all applicable to the embodiments of the readable storage medium and can achieve the same technical effects.

[0148] The embodiment of the present application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the various processes of the method embodiment of the above-mentioned video processing method are implemented, and the same technical effect can be achieved. To avoid repetition, they are not repeated here.

[0149] It should be noted that many functional components described in this specification are referred to as modules in order to more particularly emphasize the independence of their implementation methods.

[0150] In the embodiment of the present application, module can be implemented with software so that it can be executed by various types of processors. For example, an executable code module of an identification can include one or more physical or logical blocks of computer instructions, for example, it can be constructed as an object, process or function. Nevertheless, the executable code of the identified module does not need to be physically located together, but can include different instructions stored in different positions, and when these instructions are logically combined together, it constitutes a module and realizes the specified purpose of the module.

[0151] In fact, executable code module can be a single instruction or many instructions, and can even be distributed on a plurality of different code segments, distributed among different programs, and distributed across a plurality of memory devices. Similarly, operating data can be identified in the module, and can be implemented and organized in the data structure of any appropriate type according to any appropriate form. The operating data can be collected as a single data set, or can be distributed in different locations (including on different storage devices), and can only be present on a system or network as an electronic signal at least in part.

[0152] When a module can be implemented by software, considering the level of existing hardware technology, a person skilled in the art can build a corresponding hardware circuit to implement the corresponding function of the module that can be implemented by software without considering the cost. The hardware circuit includes a conventional very large scale integration (VLSI) circuit or gate array and existing semiconductors such as logic chips, transistors, or other discrete components. The module can also be implemented by a programmable hardware device, such as a field programmable gate array, a programmable array logic, a programmable logic device, etc.

[0153] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), including a number of instructions for a terminal (which can be a mobile phone, computer, server, air conditioner, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0154] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary personnel in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A video processing method, characterized in that: include: Obtain motion parameter information of the motion layer in the target video; According to the motion parameter information, a screen cropping coefficient of the video frame of the target video is obtained; the screen cropping coefficient is used to determine the size of the cropping area of ​​the video frame; The target video is cropped according to the picture cropping coefficient.

2. The video processing method according to claim 1, characterized in that: The target video includes a first video frame and a second video frame that are adjacent to each other; The step of obtaining motion parameter information of a motion layer in a target video includes: Obtaining a first center point position of the motion layer in the first video frame and a second center point position of the motion layer in the second video frame; Obtaining displacement information of the motion layer according to the first center point position and the second center point position; According to the displacement information, motion parameter information of the motion layer in the target video is obtained.

3. The video processing method according to claim 1 or 2, characterized in that: The acquiring, according to the motion parameter information, a screen cropping coefficient of a video frame of the target video includes: The image cropping coefficient of the video frame is obtained according to the cropping amplitude and the motion parameter information.

4. The video processing method according to claim 3, characterized in that: The motion parameter information includes: acceleration information and displacement information; the acceleration information includes: a first acceleration corresponding to a first coordinate axis in a reference coordinate system, and a second acceleration corresponding to a second coordinate axis in the reference coordinate system; the displacement information is the displacement of the motion layer in the reference coordinate system within a first duration; The picture cropping coefficient includes: a first cropping coefficient corresponding to a first coordinate axis in the reference coordinate system, and a second cropping coefficient corresponding to a second coordinate axis in the reference coordinate system; The acquiring the picture cropping coefficient of the video frame according to the cropping amplitude and the motion parameter information includes: Obtaining, according to the displacement information, a first motion-related influence factor corresponding to the first coordinate axis and a second motion-related influence factor corresponding to the second coordinate axis; Obtaining the first cropping coefficient according to the cropping amplitude, the first acceleration and the first motion-related influencing factor; The second cropping coefficient is obtained according to the cropping amplitude, the second acceleration and the second motion-related influencing factor.

5. The video processing method according to claim 1 or 2, characterized in that: The step of cropping the target video according to the picture cropping coefficient includes: Determining the size of the cropping area of ​​each video frame of the target video according to the picture cropping coefficient; Determining the position of the cropping area of ​​each video frame according to the motion direction information of the motion layer; The corresponding video frame is cropped according to the size and position.

6. The video processing method according to claim 1, characterized in that: Also includes: Identify a motion layer in the target video, and use a layer portion other than the motion layer as a background layer; The origin of the reference coordinate system is determined in the background layer.

7. A video processing device, characterized in that: include: A first acquisition module is used to acquire motion parameter information of a motion layer in a target video; A second acquisition module is used to acquire a picture cropping coefficient of a video frame of the target video according to the motion parameter information; the picture cropping coefficient is used to determine the size of a cropping area of ​​the video frame; The first processing module is used to crop the target video according to the picture cropping coefficient.

8. A video processing device, comprising a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that: When the processor executes the program, the video processing method according to any one of claims 1 to 6 is implemented.

9. A readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps in the video processing method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the video processing method according to any one of claims 1 to 6.