Video recording method, device, equipment and storage medium

Abstract

The present invention relates to the field of camera technology, and discloses a video recording method, device, equipment and storage medium, which obtains a video recording instruction, controls at least one fixed camera to start video recording according to the video recording instruction, saves the video recorded by at least one camera according to a preset duration, performs preprocessing, and sends the preprocessed video content to a mobile terminal. The method realizes convenient recording of videos during exercise, and improves the safety and convenience during shooting.

Description

Technical Field

[0001] The present invention relates to the field of camera technology, and in particular to a video recording method, device, equipment and storage medium. Background Art

[0002] With the rapid development of social media and short video platforms, sharing personal experiences and journeys has become a popular trend. Especially among outdoor activity and cycling enthusiasts, recording and sharing the scenery and experiences during cycling has become very common. This sharing not only increases people's interest in outdoor activities, but also promotes the development of tourism and sports culture. However, there are certain safety risks in shooting videos while riding. Riders may be distracted when operating the camera equipment, thereby increasing the risk of accidents. In addition, existing shooting equipment often needs to be handheld or body-fixed, which may be inconvenient when riding, and is easily affected by environmental factors such as motion jitter, wind noise, low-light environment, etc., affecting the video quality. Summary of the invention

[0003] In view of this, the present application provides a video recording method to solve the technical problems of safety hazards and inconvenient shooting in the existing cycling video shooting technology.

[0004] According to a first aspect of the present application, a video recording method is provided, comprising:

[0005] Get video recording instructions;

[0006] According to the video recording instruction, controlling at least one fixedly installed camera to start video recording;

[0007] Saving the video recorded by at least one camera according to a preset duration, and performing preprocessing, wherein the preprocessing includes anti-shake processing and enhancement processing;

[0008] The pre-processed video content is sent to the mobile terminal.

[0009] According to a second aspect of the present application, a video recording device is provided, comprising:

[0010] An acquisition module is used to obtain video recording instructions;

[0011] A recording module, configured to control at least one fixedly installed camera to start video recording according to the video recording instruction;

[0012] A processing module, used to save the video recorded by at least one camera according to a preset duration and perform preprocessing, wherein the preprocessing includes anti-shake processing and enhancement processing;

[0013] The sending module is used to send the preprocessed video content to the mobile terminal.

[0014] According to a third aspect of the present application, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned video recording method when executing the computer program.

[0015] According to a fourth aspect of the present application, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned video recording method are implemented.

[0016] By means of the above technical solution, the present application provides a video recording method, device, equipment and medium, which obtains a video recording instruction, controls at least one fixedly installed camera to start video recording according to the video recording instruction, saves the video recorded by at least one camera according to the preset duration, performs preprocessing, and sends the preprocessed video content to the mobile terminal. Convenient recording of videos during exercise is achieved, and the safety and convenience of the shooting process are improved. The preprocessed video content is sent to the mobile terminal, which is convenient for quick social sharing and interaction using the post-editing function of the mobile terminal.

[0017] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application and to implement it in accordance with the contents of the specification, and to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 A schematic diagram of an application scenario of a video recording method provided in an embodiment of the present application is shown;

[0020] Figure 2 A flowchart of a video recording method provided in Embodiment 1 of the present application is shown;

[0021] Figure 3 A schematic diagram of a video preprocessing process provided in Embodiment 2 of the present application is shown;

[0022] Figure 4 A schematic structural diagram of a video recording device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0023] The specific implementation of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0024] A video recording method provided by an embodiment of the present invention can be applied to Figure 1 The hardware system can be installed in a variety of transportation tools, such as bicycles, electric bicycles, motorcycles, motor vehicles, etc. The processor can be shared with the processor of the transportation tool or a separate processor can be set. The processor is connected to at least one camera installed on the vehicle (multiple cameras can realize video switching with different perspectives), the gimbal can be used to control the shooting perspective of a single camera, and the memory can store the captured video locally (can be achieved through SD card, TF card). The processor is connected to the user's mobile phone via Bluetooth or local WiFi to facilitate the transmission of the captured video. APPs on the mobile phone (such as Jianying, Douyin, Kuaishou, Xiaohongshu, etc.) can realize post-editing and social sharing of videos, and can also upload videos to the cloud. Through Figure 1 The system shown enables users to conveniently shoot travel videos and photos during travel. By obtaining video recording instructions, at least one fixed camera is controlled to start video recording according to the video recording instructions. According to the preset duration, the video recorded by at least one camera is saved, and pre-processed, and the pre-processed video content is sent to the mobile terminal. Convenient video recording during exercise is achieved, which improves the safety and convenience during shooting. The pre-processed video content is sent to the mobile terminal, which is convenient for quick social sharing and interaction using the post-editing function of the mobile terminal. For example, taking a user driving an electric bicycle as an example, the user can press the one-touch shooting button on the handlebar during riding to shoot the scenery along the way. The single shot duration is 30S or 60S. After the video is shot, it is automatically uploaded to the mobile phone APP. The user can edit, edit and dub in the APP, and can also forward it to WeChat Moments or Douyin (unedited version) to share the fun of riding. After the video is recorded, it is processed by anti-shake and enhancement.

[0025] The present invention is described in detail below through specific embodiments.

[0026] Embodiment 1

[0027] like Figure 2 As shown, a video recording method provided in an embodiment of the present invention includes:

[0028] Step 201: Obtain a video recording instruction;

[0029] Step 202: According to the video recording instruction, control at least one fixedly installed camera to start video recording;

[0030] Step 203: saving the video recorded by at least one camera according to a preset duration, and performing pre-processing;

[0031] Wherein, the preprocessing includes anti-shake processing and enhancement processing;

[0032] Step 204: Send the preprocessed video content to the mobile terminal.

[0033] The video recording processing method provided in the first embodiment of the present invention obtains a video recording instruction, controls at least one fixedly installed camera to start video recording according to the video recording instruction, saves the video recorded by at least one camera according to a preset duration, performs preprocessing, and sends the preprocessed video content to a mobile terminal. Convenient recording of videos during exercise is achieved, the safety and convenience of the shooting process are improved, and the preprocessed video content is sent to the mobile terminal, which is convenient for quick social sharing and interaction using the post-editing function of the mobile terminal.

[0034] Embodiment 2

[0035] For the first embodiment, the video is shot during the movement, and the video often shakes due to the bumps on the road, and the video is blurred and dark due to weather, illumination, etc. Therefore, it is necessary to pre-process the recorded video. The pre-processing process is as follows: Figure 3 As shown, it includes video stabilization and image enhancement (such as noise reduction, low illumination recovery, defogging, deblurring, sharpening, etc.). Accordingly, step 203 specifically includes the following steps:

[0036] Step 203-1, saving a video recorded by at least one camera according to a preset duration;

[0037] Step 203-2, performing anti-shake processing on the stored video to generate first video data;

[0038] Step 203-3: Perform noise reduction processing on the first video data to generate second video data.

[0039] The second embodiment of the present invention improves the quality of the video by performing a series of preprocessing on the recorded video, provides better video materials for later editing and sharing, and reduces the difficulty of improving the quality of the video in the later stage.

[0040] Embodiment 3

[0041] For step 203-2, since the motion smoothing method in the prior art cannot accurately distinguish between smooth movement and random jitter in the video (such as sudden jitter caused by bumps on the road), the jitter cannot be effectively suppressed during the smoothing process. In addition, the algorithm in the prior art will over-smooth the video, resulting in distortion of the video content, such as motion blur, loss of image details, and even unnatural visual effects. Embodiment 2 of the present invention proposes a method based on low-rank constraints and trajectory optimization to effectively eliminate jitter in the video and generate a stable video. Video stabilization processing includes three stages: motion estimation, motion smoothing, and motion compensation. In the motion smoothing stage, a new optimization model combining low-rank constraints and predicted path smoothing terms is used, and a variable-length local window strategy is used to optimize the motion trajectory estimation to improve the stability of the video. Accordingly, step 203-2 specifically includes:

[0042] Step 203-21, splitting the saved video into video frame sequences;

[0043] Step 203-22, meshing the video frames of the video frame sequence, and estimating the original motion trajectory of each mesh vertex based on the MeshFlow model;

[0044] Among them, step 203-22 specifically includes:

[0045] Step 203-22-1, preprocessing the input video frame sequence;

[0046] Step 203-22-2, gridding each video frame to divide the image of each video frame into a grid (for example, a regular grid of 16x16);

[0047] Step 203-22-3: Estimate the original motion trajectory of each mesh vertex based on the MeshFlow model.

[0048] Among them, step 203-22-3 is the mesh optical flow MeshFlow motion model implementation steps including:

[0049] Get MV (Motion Vector): Feature extraction image, get the feature corner point set of the image (get FAST or SIFT or ORB feature corner point set through specific method functions), the difference of feature positions matched by adjacent frames is used as MV, the feature difference is also the motion vector, which reflects the motion trend of the following frame on the previous frame, for a pair of feature matching points P and With motion vector

[0050] Get MeshFlow (i.e. mesh vertex flow), associate MV to adjacent mesh vertices, form an elliptical area at the location of the current feature point, the area contains n mesh areas (n is a positive integer), and assign the vertex of each mesh area to the motion vector of the feature point;

[0051] The first median filter F1 is used for filtering, where a mesh vertex is associated with multiple feature-matched MVs, and F1 is used for smoothing to obtain a regularized mesh vertex stream;

[0052] Perform filtering through a second median filter F2, for example, through a 3X3 median filter, to remove outlier noise and generate a spatially sparse motion field;

[0053] Vertex motion path generation: Each corresponding vertex MV is connected to form a path as the original motion trajectory.

[0054] This step has a better effect when processing large parallax scenes through the MeshFlow algorithm because it can better handle fast camera motion and large shaking.

[0055] Step 203-23, generating a smooth motion trajectory of the mesh vertices according to the original motion trajectory;

[0056] Among them, step 203-23 ​​specifically includes:

[0057] Step 203-23-1, each round of iteration calculation updates the smooth motion trajectory matrix P = (CW T +βQE T +Z+ρ(M+D)+Y+εB)·(WW T +αHH T +βEE T +(ρ+ε)I) -1 , where matrix W and matrix H are the weight matrix and difference matrix obtained by motion estimation, matrix Q contains the smooth trajectory information of the previous frame, matrix E is the selection matrix, matrix Z and matrix Y are Lagrange multiplier matrices, matrix D is the rank constraint term, B is the matrix related to the low-rank constraint, α and β are weight parameters, ρ and ε are regularization parameters, C is the original motion trajectory matrix, M is the low-rank approximation of the original motion trajectory matrix C (matrix M represents the low-rank approximation of the original motion trajectory matrix C. In the context of low-rank constraints, matrix M is the main motion information extracted from matrix C by singular value decomposition. This low-rank approximation helps to guide the smooth motion trajectory matrix P to a smooth and low-complexity solution during the optimization process), and I is a bit matrix;

[0058] Among them, this step constructs an optimized regularization model to obtain a smooth motion trajectory by optimizing the estimated motion trajectory. The model contains multiple constraints, such as low-rank constraints, predicted path smoothness constraints, and motion smoothness constraints. Finally, the optimization of the model is solved by the alternating direction multiplier method (ADMM) algorithm. In order to more clearly explain the update of the smooth motion trajectory matrix, a detailed description is given as follows:

[0059] The objective function usually includes data terms, motion smoothing terms, low-rank constraint terms, and prediction path smoothness terms, and the combination is as follows:

[0060]

[0061] in, It means that we are looking for the values ​​of matrix P, matrix B, and matrix D that minimize the objective function; is a data item, which is the square of the Frobenius norm (Frobenius norm is a commonly used matrix norm calculation, which is the square root of the sum of the squares of the matrix elements) between the original motion trajectory matrix C and the smoothed motion trajectory matrix P weighted by the weight matrix W (equivalent to the sum of the squares of the matrix elements); is the square of the Frobenius norm of the product of matrix P and matrix H, which is the motion smoothing term, where H is a difference matrix used to calculate the smoothness of the motion trajectory; is the prediction path smoothness term, which is used to ensure the consistency of motion between adjacent frames; τ||B|| * is a low-rank constraint term, where B is a matrix related to the low-rank constraint, is the nuclear norm of B, and is used to control the rank of B; λ||D||1 is a 1-norm constraint term, where D represents the difference between the smoothed trajectory and the original trajectory, and is used for regularization;<Z,M+D-P> and<Y,B-P> is the Lagrange multiplier term, Z and Y are Lagrange multipliers used to handle constraints; is the penalty term, where ρ and ε are regularization parameters used to strengthen the constraints. Based on the ADMM algorithm, the above objective function is decomposed into a series of sub-problems, each of which is optimized for P, B, and D. In each iteration, these sub-problems are solved alternately until the convergence conditions are met. In this way, the algorithm can gradually approach the optimal solution, thereby effectively solving the optimization problem with constraints. Under the ADMM algorithm, the update of the matrix P can be obtained by the formula P = (CW T +βQE T +Z+ρ(M+D)+Y+εB)·(WW T +αHH T +βEE T +(ρ+ε)I) -1The matrix W and matrix H are the weight matrix and difference matrix obtained by motion estimation, the matrix Q and matrix E are related to the smoothness of the predicted path, the matrix Z and matrix Y are Lagrange multipliers, the matrix D is a low-rank constraint term (the matrix Q is usually used to represent the smooth motion trajectory of the previous frame that has been calculated in the optimization model. The matrix Q contains the smooth trajectory information of the previous frame (or the previous frames). This information is used to calculate the motion compensation value between adjacent frames and to ensure the continuity of motion in the predicted path smoothness term), and the matrix W is a weight matrix (the matrix W can be used to assign weights to different motion trajectories or mesh vertices. During the optimization process, these weights can help the algorithm distinguish which parts of the motion trajectory are more important, such as the scenery, mountains, lakes, etc. that appear during riding, so as to give more attention during smoothing; smoothness of motion trajectory: in the motion smoothing step, the matrix W can be associated with the motion trajectory matrix P to calculate the smoothed motion trajectory; the weight matrix W can help the algorithm assign different degrees of smoothness according to different areas of the video content when smoothing the motion trajectory; the matrix W can also be used to adjust the weights of different constraints, such as low-rank constraints or predicted path smoothness. Smoothness term, W matrix helps to balance different objectives in the optimization model to achieve better stabilization effect; in the motion compensation step, matrix W can be used to calculate the motion compensation value. Based on the difference between the smoothed motion trajectory and the original motion trajectory, W matrix is ​​used to adjust the compensation value to ensure a smooth transition between video frames. In the actual application of this embodiment, the specific form and calculation method of matrix W need to be determined according to the design of the algorithm and the specific needs of video stabilization. Matrix W can be pre-defined or dynamically calculated during the iteration of the algorithm. Matrix W plays a key role in the video stabilization algorithm, ensuring the smooth processing of the motion trajectory and the improvement of video quality), matrix E is a selection matrix (matrix E is a selection matrix, which is used to extract specific columns from matrix P. Matrix E is usually used to select columns corresponding to matrix Q so as to calculate the motion difference between adjacent frames during the optimization process. The selection operation helps to maintain the smoothness and consistency of the motion trajectory during the iteration process), matrix B is a matrix related to the low-rank constraint, ρ and ∈ are regularization parameters, I is the unit matrix, matrix C is the original local motion trajectory matrix, and M=linspace(C i-n ,C i+n ,2n+1) to represent the main camera motion in the local window, where C i represents the element motion trajectory matrix. The function linspace(·) is equivalent to evenly dividing the line segment between the start and end trajectories in the local window into 2n+1 segments. Correspondingly, the original motion trajectory matrix can be expressed as C = [C i-n ,…,C i ,…,C i+n. To integrate the acceleration motion within a local window and introduce motion diversity, a smooth motion trajectory is obtained by combining the primary motion represented by M and the additional motion. P is generated by constructing the additional motion using M. Camera jitter increases the rank of the motion trajectory matrix. Therefore, the rank of the smooth local motion trajectory, rank(P), is lower than that of the original local motion trajectory, i.e., rank(P) < rank(C). ADMM (Alternating Direction Method of Multipliers) is used to handle the optimization problem with low-rank constraint and prediction path smoothness constraint. In the third embodiment of the present invention, an optimization model including a data term, a motion smoothness term, a low-rank constraint term, and a prediction path smoothness term is constructed. This model aims to minimize the difference between the original motion trajectory and the smooth motion trajectory while satisfying the low-rank constraint and maintaining the coherence of the motion between adjacent frames. In each iteration, first, B and D are fixed, and P is optimized. Then, P is fixed, and B and D are updated. This process is alternated until convergence. Lagrange multipliers (Z and Y) are used to handle the coupling between P and B. In each iteration, these multipliers are updated according to the current values of P and B. During the iteration process, it is checked whether the change amounts of P, B, and D are less than a preset threshold or whether the maximum number of iterations has been reached. If the convergence condition is satisfied, the iteration is stopped. When the ADMM algorithm converges, the obtained P matrix represents the smoothed motion trajectory, which is used for subsequent motion compensation steps to generate a stable video frame sequence. The application of ADMM enables the video stabilization algorithm to effectively handle complex motion trajectories while maintaining the coherence and naturalness of the video content. By this method, the algorithm can reduce jitter while trying to retain the important dynamic features in the video.

[0062] Step 203-23-2: Perform singular value decomposition on the matrix (P - Y / ε), apply soft thresholding to update the singular values, and reconstruct to generate the updated matrix B = U∑ * V T , where U is the left singular matrix, V is the right singular matrix, and ∑ * are the singular values after soft thresholding;

[0063] Among them, in each iteration, when P (the smoothed motion trajectory matrix) is updated, B needs to be updated to satisfy the low-rank constraint. First, perform singular value decomposition (SVD) on P to obtain U, Σ singular values, and V T . Then apply soft thresholding to update Σ, and finally reconstruct B = U * Σ * V T. The soft threshold operation is to achieve low-rank approximation by setting a threshold τ and setting the part of the singular value in Σ that is less than τ to 0. Soft thresholding is a mathematical tool commonly used in signal processing and statistics, especially when dealing with sparse data and performing regularization. In the context of video stabilization algorithms, soft thresholding is used to implement low-rank constraints, that is, to simplify the complexity of the motion trajectory by reducing the rank of the matrix, such as selecting a threshold parameter (usually expressed as λ or τ). This parameter determines which elements will be retained and which will be set to zero when performing a soft thresholding operation. For example, for each element in the matrix A, perform soft threshold (A) Operation:

[0064] If the absolute value of the element is greater than the threshold threshold, retain the element but subtract the threshold threshold (ie, element value = original value - threshold threshold); if the absolute value of the element is less than or equal to threshold, set the element to zero.

[0065] Soft threshold processing is part of the regularization technique, which is very important in many optimization problems, especially when high-dimensional data needs to be processed or noise needs to be suppressed. Soft threshold processing can reduce the number of non-zero elements in the matrix, thereby achieving sparsification. This helps to reduce the rank of the matrix, because the rank is determined by the number of linearly independent columns (or rows) in the matrix. The soft threshold processing in the embodiment of the present invention is usually used to process the matrix B or D to achieve low-rank constraints. This helps to reduce unnecessary motion information while smoothing the motion trajectory, thereby improving the stability of the video while maintaining the coherence and naturalness of the video content.

[0066] Step 203-23-3, perform soft thresholding on the matrix (PZ / ρ-M) to update the matrix D;

[0067] In each iteration, D needs to be updated to reflect the difference between P and M (the original motion trajectory matrix). in is a soft threshold operation, λ is the parameter of the low-rank constraint, and ρ is the penalty parameter in the ADMM algorithm. This step ensures that D represents the difference between P and M, and controls the size of this difference through soft threshold processing.

[0068] Step 203-23-4, update the matrix Y and the matrix Z according to Y=Y+ε(BP) and Z=Z+ρ(M+DP);

[0069] Among them, in each iteration, Z needs to be updated to reflect the change of the Lagrange multiplier, Z = Z + ρ (M + DP), the update of Z reflects the difference between P and M and the change of D related to the low-rank constraint, ρ is the penalty parameter in the ADMM algorithm, which determines the strength of the constraint. In each iteration, Y needs to be updated to reflect the constraint relationship between B and P Y = Y + ε (BP), the update of Y reflects the difference between B and P, ε is another penalty parameter in the ADMM algorithm, which determines the strength of the constraint.

[0070] Step 203-23-5, check whether the change of the updated matrix P is less than the preset threshold or whether the maximum number of iterations has been reached. When the change of the updated matrix P is less than the preset threshold or the maximum number of iterations has been reached, stop the iterative calculation and output the smooth motion trajectory of the mesh vertices.

[0071] Among them, by repeatedly applying the update steps of the optimization model, the estimate of the motion trajectory matrix P is gradually improved until a stable state is reached. After each iteration, the algorithm checks whether the convergence conditions are met, which usually involves comparing the changes in variables (such as P, B, D) between consecutive iterations. If these changes are less than a preset threshold (for example, 0.0015), the algorithm is considered to have converged. Inside the loop, the algorithm updates the matrices P, B, D, Z and Y. These updates are based on the current estimate and the constraints of the optimization model. When the change in the matrix update between consecutive iterations is small enough, or when the preset number of iterations is reached, the loop terminates, indicating that a solution that satisfies the optimization problem has been found, or the iteration limit of the algorithm has been reached. The loop iteration process is the main step of the ADMM algorithm, which ensures that in the solution process, by alternating optimization and updating of constraints, a solution that satisfies all constraints is finally found. In the video anti-shake preprocessing of this embodiment, it means that a smooth and coherent video trajectory can be found, thereby generating a stable video frame sequence.

[0072] Step 203-24: Calculate the motion compensation value of each video frame according to the difference between the smooth motion trajectory of the mesh vertex and the original motion trajectory, and perform mesh deformation transformation on each video frame according to the motion compensation value to generate first video data.

[0073] In order to have a clearer understanding of the entire video stabilization preprocessing process, it is summarized as follows:

[0074] Initialization: At the beginning of the algorithm, the matrix P (the smoothed motion trajectory matrix) is usually initialized. This can be done by setting P to the same value as the original motion trajectory matrix C, or by estimating the initial smooth trajectory through some heuristic method.

[0075] Iterative update: In each iteration, the matrix P is updated according to the results of the optimization model. This optimization model can include data terms (making the smooth trajectory as close to the original trajectory as possible), motion smoothing terms (reducing the jitter of the trajectory), low-rank constraint terms (reducing the complexity of the trajectory), and predicted path smoothness terms (ensuring the coherence of the trajectory between adjacent frames).

[0076] Solve the optimization problem: Use ADMM (Alternating Direction Method of Multipliers) or other optimization algorithms to solve the optimization problem with the above constraints. This usually involves iterative updates of matrices P, B, D, Z, and Y until convergence conditions are met.

[0077] Convergence condition judgment: After each iteration, check whether the convergence condition is met. This may include checking whether the change in the matrix P between consecutive iterations is less than a preset threshold, or whether the maximum number of iterations has been reached.

[0078] Output smooth trajectory: When the algorithm converges, the output matrix P is used as the final smooth motion trajectory. These trajectories will be used in the subsequent motion compensation step to generate a stable video frame sequence.

[0079] Motion compensation: The smoothed trajectory in the smoothed motion trajectory matrix P is used to calculate the motion compensation value, which is applied to the video frame to eliminate jitter and generate a stable video output.

[0080] In addition, in order to improve the smoothness of the motion trajectory, a local window strategy is adopted. A local window is constructed for each video frame, and the video frame in the local window is meshed accordingly, and the original motion trajectory of each mesh vertex is estimated based on the MeshFlow model. On the original motion trajectory, a local window centered on the i-th frame (i is a natural number that can represent each video frame) is constructed, which contains the motion trajectory of the adjacent frames. The local window strategy smoothes the fast motion mode by adaptively shortening the window length, avoiding the problem of over-cropping of the image due to over-smoothing. This strategy achieves a better balance between maintaining motion smoothness and retaining necessary image details. The window length is dynamically adjusted for the size of the local window. When fast motion is detected, the window length will be dynamically adjusted. For example, when the motion changes dramatically, using a smaller local window can better fit the original motion trajectory, thereby avoiding over-smoothing and loss of details.

[0081] Embodiment 4

[0082] In order to reduce the amount of calculation in the video preprocessing process, the mechanical anti-shake of the pan / tilt can be used to reduce the jitter of the camera lens in the early stage of the recording process, and the anti-shake algorithm of the third embodiment can be used for further processing after the stability of the video is improved. A mechanical anti-shake control algorithm is introduced in the fourth embodiment of the present invention. Two-axis or three-axis mechanical anti-shake is often used in the prior art. Its basic workflow includes attitude angle solution, control solution and motor drive. The control algorithm anti-shake is limited by hardware conditions such as the number of axes and whether the IMU is installed. For low-cost methods, a two-axis pan / tilt is usually installed on an electric bicycle. The mechanical anti-shake effect of the two-axis pan / tilt is not as good as that of the three-axis. Therefore, the video anti-shake can be combined with the third embodiment to enhance the anti-shake effect. During the use of the pan / tilt, it is inevitable that the electric bicycle will have a sudden change in motion state (for example, potholes or stones on the road), causing the pan / tilt to rotate to the limit and collide, such as running, violent shaking or falling, etc. This collision will cause the angular velocity data measured by the sensor to be discontinuous, and a large jump value will appear, causing the picture to shake. This embodiment is optimized on the basis of the existing mechanical anti-shake algorithm, and detects the occurrence of collision so as to perform rapid recovery. Therefore, during the video recording process in step 202, it is necessary to detect in real time whether a collision occurs and restore the state of the pan / tilt in time, so the following steps are included:

[0083] Step 401, detecting whether a collision occurs;

[0084] The detection algorithm integrates the modulus of the angular velocity within a preset time. When the value exceeds a certain threshold, it is considered that a collision exists in this stage.

[0085] Step 402: when a collision is detected, the attitude angle of the gimbal is obtained and the quaternion is calculated;

[0086] Among them, get the attitude angle [θ0, β0, α0] at the time of collision and calculate the corresponding quaternion Use it as the initialization value for subsequent iterative calculations. The iterative formula is: ω x ,ω y ,ω z is the three-dimensional data of the angular velocity in the x, y, and z axes, and Δt is the sampling period, for example, 0.001

[0087] Step 403, calculating the error value between the gravity direction vector and the measured gravity direction vector;

[0088] Among them, calculate the gravity direction vector j0,j1,j2,j3 are J k The average value of each row vector in the error value is E r =Gg is the measured gravity direction vector.

[0089] Step 404: update the angular velocity three-dimensional data according to the error value;

[0090] Among them, P control is used to continuously reduce the error value and update the angular velocity three-dimensional data λ is a control parameter and its value can be determined based on experience.

[0091] Step 405: determine whether the difference between the new angular velocity three-dimensional data and the angular velocity three-dimensional data before updating is less than a preset threshold;

[0092] Among them, judgment |J k -J k-1 |<J threshold , J threshold Can be based on experience

[0093] Step 406: When the difference between the new angular velocity three-dimensional data and the angular velocity three-dimensional data before updating is less than a preset threshold, the attitude angle is corrected according to the new angular velocity three-dimensional data.

[0094] In the fourth embodiment of the present invention, after a collision, the gimbal is in a stable state for a short time, and a correction value is obtained by combining the three-axis acceleration value with the actual characteristic that the gravity direction is downward. The correction value is subjected to multiple PI controls to continuously reduce the error of the attitude angle, thereby achieving a rapid recovery effect without the need for human intervention and continuing to work normally.

[0095] Further, as Figure 2 In the specific implementation of the method, a video recording device is provided in an embodiment of the present invention, such as Figure 4 As shown, the device comprises:

[0096] An acquisition module 410 is used to acquire a video recording instruction;

[0097] The recording module 420 is used to control at least one fixedly installed camera to start video recording according to the video recording instruction;

[0098] The processing module 430 is used to save the video recorded by at least one camera according to a preset time length and perform preprocessing, wherein the preprocessing includes anti-shake processing and enhancement processing;

[0099] The sending module 440 is used to send the pre-processed video content to the mobile terminal.

[0100] In an embodiment of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of implementing the video recording method include:

[0101] Get video recording instructions;

[0102] According to the video recording instruction, controlling at least one fixedly installed camera to start video recording;

[0103] Saving the video recorded by at least one camera according to a preset duration, and performing preprocessing, wherein the preprocessing includes anti-shake processing and enhancement processing;

[0104] The pre-processed video content is sent to the mobile terminal.

[0105] In an embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0106] Get video recording instructions;

[0107] According to the video recording instruction, controlling at least one fixedly installed camera to start video recording;

[0108] Saving the video recorded by at least one camera according to a preset duration, and performing preprocessing, wherein the preprocessing includes anti-shake processing and enhancement processing;

[0109] The pre-processed video content is sent to the mobile terminal.

[0110] It should be noted that the above embodiments only use electric bicycles to illustrate the principles and implementation steps of the embodiments of the present invention, and no specific limitation is made to the actual means of transportation. For example, the technical solution of the present invention can also be applied to electric bicycles, motorcycles, cars, etc. For the functions or steps that can be implemented by computer-readable storage media or computer devices, please refer to the aforementioned method embodiments. To avoid repetition, they will not be described one by one here.

[0111] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0112] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0113] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A video recording method, characterized in that: include: Get video recording instructions; According to the video recording instruction, controlling at least one fixedly installed camera to start video recording; Save the video recorded by at least one camera according to the preset duration; Split the saved video into video frame sequences; Meshing the video frames of the video frame sequence, and estimating the original motion trajectory of each mesh vertex based on the MeshFlow model; Each round of iteration calculates and updates the smooth motion trajectory matrix P=(CW T +βQE T +Z+ρ(M+D)+Y+εB)·(WW T +αHH T +βEE T +(ρ+ε)I) -1 , where matrix W and matrix H are the weight matrix and difference matrix obtained by motion estimation, matrix Q contains the smooth trajectory information of the previous frame, matrix E is the selection matrix, matrix Z and matrix Y are Lagrange multiplier matrices, matrix D is the rank constraint term, B is the matrix related to the low-rank constraint, α and β are weight parameters, ρ and ε are regularization parameters, C is the original motion trajectory matrix, M is the low-rank approximation of the original motion trajectory matrix C, and I is the bit matrix; Perform singular value decomposition on the matrix (PY / ε), apply soft thresholding to update the singular values, and reconstruct the updated matrix B=U∑ * V T , where U is the left singular matrix, V is the right singular matrix, ∑ * is the singular value after soft thresholding; Perform soft thresholding on the matrix (PZ / ρ-M) to update the matrix D; Update the matrix Y and the matrix Z according to Y=Y+ε(BP) and Z=Z+ρ(M+DP); Check whether the change of the updated matrix P is less than the preset threshold or whether the maximum number of iterations has been reached. When the change of the updated matrix P is less than the preset threshold or the maximum number of iterations has been reached, stop the iterative calculation and output the smooth motion trajectory of the mesh vertices; Calculating a motion compensation value for each video frame according to a difference between the smooth motion trajectory of the mesh vertex and the original motion trajectory, and performing a mesh deformation transformation on each video frame according to the motion compensation value to generate first video data; Performing noise reduction processing on the first video data to generate second video data; The pre-processed video content is sent to the mobile terminal.

2. The video recording method according to claim 1, characterized in that: The steps of meshing the video frames of the video frame sequence and estimating the original motion trajectory of each mesh vertex based on the MeshFlow model include: Adopting a local window strategy, constructing a local window for each video frame, wherein the local window includes the current video frame and a preset number of adjacent video frames, and the preset number is the size of the local window; The video frame in the local window is meshed, and the original motion trajectory of each mesh vertex is estimated based on the MeshFlow model.

3. The video recording method according to claim 2, characterized in that: The step of meshing the video frame in the local window and estimating the original motion trajectory of each mesh vertex based on the MeshFlow model includes: Preprocess the input video frame sequence; Performing gridding processing on each video frame to divide the image of each video frame into grids; Extract key feature points of each grid; Tracking the key feature points to estimate the motion trajectory of the key feature points; The motion trajectories of all feature points of each mesh are integrated into the corresponding mesh to obtain the original motion trajectory of each mesh vertex.

4. The video recording method according to claim 2, characterized in that: The local window strategy includes: The size of the local window is dynamically adjusted according to the movement speed.

5. A video recording device, characterized in that: include: An acquisition module is used to obtain video recording instructions; A recording module, configured to control at least one fixedly installed camera to start video recording according to the video recording instruction; The processing module is specifically used to save the video recorded by at least one camera according to a preset duration; split the saved video into a video frame sequence; mesh the video frames of the video frame sequence, and estimate the original motion trajectory of each mesh vertex based on the MeshFlow model; and update the smooth motion trajectory matrix in each round of iterative calculation. P=(CW T +βQE T +Z+ρ(M+D)+Y+εB)·(WW T +αHH T +βEE T +(ρ+ε)I) -1 , where matrix W and matrix H are the weight matrix and difference matrix obtained by motion estimation, matrix Q contains the smooth trajectory information of the previous frame, matrix E is the selection matrix, matrix Z and matrix Y are Lagrange multiplier matrices, matrix D is the rank constraint term, B is the matrix related to the low-rank constraint, α and β are weight parameters, ρ and ε are regularization parameters, C is the original motion trajectory matrix, M is the low-rank approximation of the original motion trajectory matrix C, and I is the bit matrix; perform singular value decomposition on the matrix (PY / ε), apply soft thresholding to update the singular values, and reconstruct the updated matrix B=U∑ * V T , where U is the left singular matrix, V is the right singular matrix, ∑ * is a singular value after soft thresholding; soft thresholding is performed on the matrix (PZ / ρ-M) to update the matrix D; the matrix Y and the matrix Z are updated according to Y=Y+ε(BP) and Z=Z+ρ(M+DP); it is checked whether the change of the updated matrix P is less than a preset threshold or whether the maximum number of iterations has been reached, when the change of the updated matrix P is less than the preset threshold or the maximum number of iterations has been reached, the iterative calculation is stopped, and the smooth motion trajectory of the mesh vertices is output; according to the difference between the smooth motion trajectory of the mesh vertices and the original motion trajectory, the motion compensation value of each video frame is calculated, and the mesh deformation transformation is performed on each video frame according to the motion compensation value to generate the first video data; the first video data is subjected to denoising to generate the second video data; The sending module is used to send the preprocessed video content to the mobile terminal.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the video recording method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the video recording method according to any one of claims 1 to 4 are implemented.

Images