A dynamic texture video coding method

Abstract

This invention discloses a method for encoding dynamic texture video, comprising: extracting a pixel-level motion field from an input video frame; performing fluid dynamics decomposition to obtain a laminar flow field that satisfies a rotation-free constraint and a turbulent flow field that satisfies a divergence-free constraint; encoding the laminar flow field using an affine motion model to generate laminar motion parameters; inputting the turbulent flow field into a pre-built neural ordinary differential equation network for parameterized encoding to generate turbulence parameters; and multiplexing the laminar motion parameters and turbulence parameters into an output bitstream. This method improves the temporal coherence of texture details, enhances coding efficiency, and avoids over-allocation.

Description

Technical Field

[0001] The invention belongs to the field of video coding, in particular to a dynamic texture video coding method. Background Art

[0002] Dynamic texture video coding is a core technical challenge in enhancing the immersive media experience. Traditional coding frameworks have significant performance bottlenecks in non-rigid motion scenes such as flames and water flow. The International Telecommunication Union report shows that the bit rate of the H.265 standard in dynamic texture areas accounts for as much as 68% of the total video bit rate, but the video multi-method fusion (VMAF) score is still 22.3 points lower than the original video. This inefficiency stems from the difficulty of block matching algorithms in capturing the non-stationary characteristics of turbulent motion, such as the changes in vortex structure during smoke diffusion. For videos compressed using High Efficiency Video Coding (HEVC), the fidelity of texture details is sublinearly related to the bit rate. When the bit rate exceeds 20Mbps, the visual quality still cannot reach the user-acceptable threshold.

[0003] Existing dynamic texture encoding methods can be divided into two categories: physics-driven and data-driven, but both have inherent limitations. Fluid simulation methods based on the Navier-Stokes equations (such as Stam's stable fluid algorithm) can accurately model flame motion, but single-frame calculations are time-consuming and cannot meet the requirements of real-time encoding. Purely data-driven generative models have a low correlation coefficient between the Fourier spectrum of the generated flame and the real data, indicating that the lack of physical constraints leads to statistical distortion. Hybrid methods improve the compressibility of turbulent fields, but the vorticity preservation error in long-term predictions increases exponentially with the number of frames.

[0004] The fundamental flaw of existing technologies lies in the improper decoupling of motion modeling and texture synthesis, making it difficult to achieve both high dynamic texture fidelity and computational efficiency. Traditional encoders treat turbulent motion as residual information, and using the DCT transform for quantization destroys its multi-scale structural characteristics, reducing the Kolmogorov complexity of the smoke region. While deep learning methods improve compression efficiency through end-to-end training, their latent spaces lack physical interpretability. More seriously, the pixel-level loss functions (such as MSE) used by existing methods cannot effectively assess the visual importance of dynamic textures, resulting in insufficient prioritization of high-frequency vortex structures, which are sensitive to humans. Summary of the Invention

[0005] The purpose of the invention is to provide a dynamic texture video encoding method, in order to solve at least one technical problem existing in the prior art.

[0006] Technical solution: a dynamic texture video encoding method, comprising:

[0007] Obtain an input video frame and extract a pixel-level motion field of the input video frame;

[0008] Perform fluid dynamics decomposition on the pixel-level motion field to obtain a laminar flow field that satisfies the irrotation constraint and a turbulent flow field that satisfies the divergence constraint;

[0009] Encode the laminar flow field with an affine motion model to generate laminar flow motion parameters;

[0010] The turbulence field is input into the pre-built neural ordinary differential equation network for parameterized encoding to generate turbulence parameters;

[0011] The laminar motion parameters and turbulence parameters are multiplexed into the output code stream.

[0012] Beneficial effects: The present invention improves the VMAF score of dynamic texture areas and improves the temporal coherence of texture details; reduces the single-frame calculation delay, improves coding efficiency, and avoids over-allocation. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A flowchart of the steps of a dynamic texture video encoding method provided in an embodiment of the present application.

[0014] Figure 2 A flow chart of the steps for obtaining laminar flow field and turbulent flow field provided in the embodiments of the present application.

[0015] Figure 3 A flowchart of the steps for generating turbulence parameters provided in an embodiment of the present application.

[0016] Figure 4 A flowchart of the steps for extracting the potential representation of the turbulence field provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0018] It should be noted that the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or are inherent to these processes, methods, products or apparatus.

[0019] Research has revealed that current video coding technologies face technical bottlenecks when processing dynamic textures. Traditional hybrid architectures based on block matching and transform coding (such as H.265 / HEVC) suffer from inefficient compression of textures with non-rigid motion, such as flames, water, and smoke. The fundamental problem lies in the inability of rigid motion models to accurately describe the nonlinear characteristics of turbulent motion, and the difficulty of using discrete cosine transforms to capture the random statistical properties of textures. This leads to texture collapse, motion artifacts, and detail distortion even at high bitrates. Existing methods typically mitigate these issues by increasing quantization accuracy or motion vector density, but this results in wasted resources. Furthermore, while purely data-driven deep learning coding schemes can partially improve quality, they suffer from large model parameters and high inference latency, making them incapable of meeting the requirements of real-time communication. These shortcomings collectively constitute a long-standing technical challenge in the field of dynamic texture video coding.

[0020] like Figure 1 As shown, a dynamic texture video encoding method is proposed, comprising the following steps:

[0021] Obtain an input video frame and extract a pixel-level motion field of the input video frame;

[0022] In this embodiment, the system receives a flame video sequence with a resolution of 1920×1080 and 30fps. Specifically, a dense optical flow algorithm (Farneback) is used to calculate the motion field v(x, y) between adjacent frames. The search window is set to 15×15 pixels and the number of pyramid levels is 3. For each pixel, the algorithm outputs the horizontal motion component v x and the vertical motion component v y The calculated motion field accuracy reaches sub-pixel level, with an average endpoint error (EPE) of less than 0.8 pixels.

[0023] The pixel-level motion field is decomposed by fluid dynamics to obtain a laminar flow field that satisfies the irrotation constraint and a turbulent flow field that satisfies the divergence constraint.

[0024] In this embodiment, the motion field v is decomposed by applying the Helmholtz-Hodge decomposition theorem. Specifically, by solving the Poisson equation ▽ 2 Φ = ▽·v and ▽ 2 ψ = -▽×v, where ▽ is the gradient operator, Φ is the scalar potential, and ψ is the vector potential. The laminar flow field is calculated as v lam = ▽Φ, the turbulence field is calculated as v turb = ▽×ψ. The conjugate gradient method is used to solve the problem. After 50 iterations, the residual error is reduced to 10 -6 The decomposed laminar flow field satisfies the irrotational constraint |▽×v lam | < 10 -5, the turbulence field satisfies the divergence-free constraint |▽·v turb | < 10 -5 .

[0025] The laminar flow field is encoded with an affine motion model to generate laminar motion parameters.

[0026] In this embodiment, the laminar flow field represents the overall spread trend of the flame, and a 6-parameter affine model is used for modeling. The parameters include horizontal translation t x , vertical translation t y , rotation angle θ, scale factor s, and horizontal shear k x and vertical shear k y The parameter values ​​are obtained by least square fitting. For example, the laminar motion parameters of a certain frame are [t x =3.2, t y =-1.5, θ=0.05, s=1.02, k x =0.01, k y =-0.02]. Each parameter is quantized to 16-bit fixed points, totaling 12 bytes.

[0027] The turbulence field is input into a pre-built neural ordinary differential equation network for parameterized encoding to generate turbulence parameters; the neural ordinary differential equation network models the time domain evolution characteristics of the turbulence field through continuous-time dynamic equations.

[0028] In this embodiment, the turbulent field v turb First, an 8-layer convolutional encoder is input with a kernel size of 3×3 and the number of channels in the order [64, 128, 256, 512, 512, 256, 128, 8]. The encoder outputs a potential representation h0 of 8×120×68. The neural ordinary differential equation (ODE) network adopts a 5-layer MLP structure with a hidden layer dimension of 256. By solving the differential equation dh / dt = f θ (h, t) predicts the turbulence evolution of the next 5 frames, where h is the potential representation, t is the time variable, and f θ A soft plus activation function is used to make the output smooth and differentiable. The time integration step Δt is set to 1 / 30 second, and the solution is solved using the fourth-order Runge-Kutta method. The final turbulence parameters include the initial latent representation h0 and the ODE network parameters θ. The data volume is 0.9 MB per frame.

[0029] The laminar motion parameters and turbulence parameters are multiplexed into the output code stream.

[0030] In this embodiment, the encoder combines laminar motion parameters (12 bytes) and turbulence parameters (0.9 MB) to form a bitstream. The bitstream header contains 2 bytes of identification information, indicating the group of pictures (GOP) length (90 frames) and the block size (16×16). For 1080p flame videos, this embodiment reduces the bitrate compared to traditional H.265 encoding. For example, in some embodiments, the flame area captured by the camera occupies 30% of the frame. The system detects that the main flame spreads toward the upper right, and the laminar flow field shows an overall motion vector of (5.2, -3.8) pixels / frame. Random motion, such as sparks, is captured by the turbulent field, whose energy is primarily concentrated in the 20-80 Hz frequency band. The decoder reconstructs the flame outline motion based on the laminar flow parameters and simulates the spark effect using 5000 particles based on the turbulence parameters. The particle color gradually changes from orange (RGB: 255, 140, 0) to dark red (RGB: 139, 0, 0). Through the above steps, efficient encoding of dynamic textures is achieved, which improves the VMAF score. At the same time, the relative error of the vorticity field reconstructed at the decoding end is reduced, maintaining the physical authenticity of the flame texture.

[0031] like Figure 2 As shown, according to one aspect of the present application, a laminar flow field and a turbulent flow field are obtained, including:

[0032] The Helmholtz decomposition theorem is applied to the pixel-level motion field to decompose it into potential flow component and vortex component.

[0033] In this embodiment, the motion field v(x, y) of the waterfall scene is physically decomposed. Specifically, according to the Helmholtz theorem, any vector field can be decomposed into v = ▽Φ + ▽×A, where Φ is the scalar potential and A is the vector potential. By constructing the Poisson equation ▽ 2 Φ = ▽·v, using the multigrid method, the grid level is set to 5 layers, and the finest grid resolution is the same as the original video. The convergence criterion is set to the relative residual less than 10 -6 .

[0034] The Hodge projection algorithm is used to iteratively optimize the potential flow component and the vortex component so that the potential flow component satisfies the irrotational constraint ▽×v=0, and the vortex component satisfies the divergence-free constraint ▽·v=0; where ▽ is the differential operator and v is the pixel-level motion vector field; the potential flow component that satisfies the irrotational constraint is regarded as a laminar flow field, and the vortex component that satisfies the divergence-free constraint is regarded as a turbulent field.

[0035] In this embodiment, the initial decomposition result is subjected to Hodge projection optimization so that the components meet the physical constraints. Specifically, the projection operator P is defined as lam and P turb , where P lam (v) = v - ▽×(▽×v / ▽2 ), P turb (v) = v - ▽(▽·v / ▽ 2 ). During the iteration process, two projection operators are applied alternately. The kth iteration is: the laminar flow field v lam k+1 = P lam (v -v turb k ), turbulent field v turb k+1 = P turb (v - v lam k+1 ). After 30 iterations, the curl of the laminar flow field |▽×v lam |Down to 2.1×10 -6 , the divergence of the turbulent field |▽·v turb |Down to 1.8×10 -6 .

[0036] like Figure 3 As shown, according to one aspect of the present application, generating turbulence parameters includes:

[0037] The turbulence field is input into the convolutional encoder to extract the latent representation of the turbulence field.

[0038] In this embodiment, the turbulent field v turb (x, y) is normalized to the range [-1, 1] and fed into the convolutional encoder. The encoder architecture consists of eight convolutional blocks, each consisting of a Conv3×3 convolutional layer, BatchNorm (batch normalization), and a ReLU (rectified linear unit). Downsampling is achieved through convolutions with a stride of 2, gradually reducing the feature map size from 1920×1080 to 120×68. The final layer uses the hyperbolic tangent (Tanh) activation function and outputs an 8-channel latent representation h∈R 8×120×68 .

[0039] The continuous evolution trajectory of the potential representation in the time domain is predicted through a neural ordinary differential equation network to generate turbulence parameters.

[0040] In this embodiment, the neural ODE network is designed to maintain a structure without divergence constraints. The dynamic function f θ The following architecture is used: the 8-dimensional input is mapped to a 256-dimensional latent space through an MLP, which is divided into two branches. The main branch generates the velocity field update Ξh / Ξt, and the auxiliary branch calculates the divergence ▽·(Ξh / Ξt), where Ξ is the partial derivative. The projection layer ensures that Ξh / Ξt - ▽(▽·(Ξh / Ξt) / ▽ 2 ), so that the output satisfies the divergence constraint. The network parameters θ contain 5 layers of fully connected layers, with a total of about 500,000 parameters. Given the initial potential representation h(t0), by solving the differential equation dh / dt = fθ (h, t) predicts the state at the future time. An adaptive step-size ODE solver is used with an initial step size of Δt = 0.01 seconds. Specifically, for the target time t1, h(t1) = h(t0) + ∫ t0 t1 f θ (h(τ), τ)dτ, where τ is the integral variable or intermediate time point, and d is the differential operator. During the integration process, the local truncation error is monitored and the error is calculated when it exceeds the threshold of 10 -4 For a 30fps video, the computation time for predicting 5 frames (167ms) is 12ms.

[0041] For example, in some embodiments, the turbulent field in the falling area of ​​water presents a complex vortex structure. After Helmholtz decomposition, the laminar field captures the main falling motion of the water flow (average velocity 9.8m / s), and the turbulent field includes the random motion of the splashing water. The dynamic function learned by the neural ODE network can accurately predict the diffusion pattern of the splash, and the mean square error between the predicted turbulent field after 5 frames and the true value is only 0.023. The decoding end reconstructs the water mist effect through 2 million particles based on the predicted trajectory. The initial velocity of the particles is sampled from the turbulent field, following the gravitational acceleration g=9.8m / s 2 In addition, when dealing with smoke scenes, the physical constraints further include buoyancy terms. The dynamic equation is modified to dh / dt = f θ (h, t) + β·T·e y , where β is the thermal expansion coefficient (0.003 / K), T is the temperature field, and e y is a vertical unit vector. This reduces the prediction error for the rising smoke. Through the above steps, accurate modeling of the turbulent flow field under physical constraints is achieved. Tests show that compared to unconstrained neural networks, divergence-free ODE networks reduce the rate of error accumulation per frame.

[0042] like Figure 4 As shown, according to one aspect of the present application, extracting a potential representation of a turbulent field comprises:

[0043] An 8-layer convolutional neural network is constructed to map the turbulence field into a high-dimensional feature space through layer-by-layer feature extraction.

[0044] In this embodiment, a dedicated encoder is constructed for dynamic textures of clouds and fog. The first layer Conv (3×3, 64) extracts local motion patterns with a receptive field of 3×3 pixels. The number of channels in the second to fourth layers increases [128, 256, 512] to capture the mesoscale vortex structure. The fifth and sixth layers maintain 512 channels for feature refinement. The number of channels in the seventh and eighth layers decreases [256, 128] to prepare for compression. Each layer is followed by group normalization (GroupNorm, 32 groups) and a leaky linear rectification function (LeakyReLU, α=0.2). The feature map is gradually downsampled by convolution with a step size of 2, and the spatial resolution is reduced from 1920×1080 to 120×68.

[0045] The features in the high-dimensional feature space are compressed into an 8-dimensional latent code space through progressive dimensionality reduction, generating a latent code of size 8×H / 16×W / 16, where H and W are the height and width of the input video frame respectively.

[0046] In this embodiment, the last layer uses 1×1 convolution to compress the 128-dimensional features to 8 dimensions, forming a compact representation h∈R 8×120×68 The compression process adopts the information bottleneck principle and minimizes I(h;v turb ) - βI(h;y) balances compression rate and reconstruction quality, where β = 0.01 controls this trade-off and I( ) represents the mutual information. The eight channels of the latent code correspond to motion components of different frequencies. Principal component analysis (PCA) validates this by improving the cumulative contribution of the first eight principal components.

[0047] The latent code is uniformly quantized to 8 bits to generate a quantized latent representation.

[0048] In this embodiment, the quantization process first counts the distribution of potential codes in the training set, and each channel of the potential representation h approximately obeys the mean 0 and standard deviation σ i Gaussian distribution. Set the quantization interval to [-3σ i , 3σ i ], covering 99.7% of the values. The uniform quantization formula is: h q = round((h - h min ) / (h max -h min )×255), where h q is the quantized potential representation, h min is the minimum potential representation, h max is the maximum potential representation. Quantization error is compensated by noise injection training. During training, uniform noise U(-0.5, 0.5) is added to simulate the quantization effect.

[0049] According to one aspect of the present application, the potential representation is further optimized, specifically:

[0050] The turbulence field is input into the pre-trained VGG network to extract the mid-level features and calculate the original Gram matrix.

[0051] In this embodiment, the VGG-19 network pre-trained by ImageNet is used to extract the relu3_3 layer (ReLU third layer third level activation output) feature F∈R 256×H'×W' , where H'=H / 8, W'=W / 8. The Gram matrix is ​​calculated as G ij = (1 / N)Σ k F ik ·F jk , where N=H'×W' is the normalization factor, F ik is the activation value of the i-th channel (or filter) at the k-th pixel. This layer is selected based on receptive field analysis. The receptive field of relu3_3 (40×40 pixels) just matches the characteristic scale of the cloud texture. The original Gram matrix G orig ∈R 256×256 Encodes the second-order statistical properties of the texture.

[0052] The turbulence field is reconstructed according to the quantized potential representation, and its Gram matrix is ​​calculated to obtain the reconstructed Gram matrix.

[0053] In this embodiment, the decoder adopts the mirror structure of the encoder, and the 8-dimensional quantization parameter h q Restore the turbulence field v' turb The reconstructed turbulence field is also input into VGG-19 to extract features and calculate the reconstructed Gram matrix G. rec .

[0054] The quantized latent representation is optimized by minimizing the difference between the original Gram matrix and the reconstructed Gram matrix to obtain the optimized latent representation.

[0055] In this embodiment, the Gram loss L is defined as gram = ||G orig -G rec || F 2 , where ||·|| F is the Frobenius norm. The total loss function is L total = L pixel + λ gram ×L gram , where L pixel = ||v turb - v'turb ||2 2 is the pixel-level reconstruction loss, λ gram =0.7 is the weight coefficient. Adam optimizer is used (learning rate lr=3×10 -4 , first-order momentum decay factor β1=0.9, second-order momentum decay factor β2=0.999) iterate 1000 steps to make L gram Dropped to below 0.01.

[0056] For example, in some embodiments, the original turbulent field presents a multi-scale vortex structure. The diagonal elements G of the Gram matrix of the VGG feature ii Reflects the energy of each channel, the non-diagonal element G ij (i≠j) represents the correlation between channels. The Gram matrix before quantization shows that the correlation coefficient between the 23rd and 89th channels reaches 0.82, corresponding to the spiral rising pattern of the smoke. The optimized quantization parameters maintain this correlation (0.79), ensuring the visual authenticity of the reconstructed smoke. In addition, for periodic water wave textures, the Gram matrix optimization is extended to multiple scales. In addition to relu3_3, the relu2_2 and relu4_4 layer features are also extracted to calculate the multi-scale Gram loss L gram = Σ l w l ||G orig l -G rec l || F 2 , weight w l = [0.3, 0.5, 0.2], where l represents different VGG network levels. This improves the Structural Similarity Index (SSIM) of high-frequency details in water waves. Through these steps, the statistical properties of textures are accurately preserved. Comparative tests show that adding the Gram matrix constraint improves the user subjective score (MOS), texture naturalness score, and compression rate.

[0057] According to one aspect of the present application, multiplexing laminar motion parameters and turbulence parameters into an output bitstream includes a closed-loop rate control mechanism:

[0058] The code rates of laminar motion parameters and turbulence parameters are evaluated separately.

[0059] In this embodiment, the amount of data output by the encoder is monitored in real time. The laminar flow motion parameters use a 6-parameter affine model, each macroblock occupies 12 bytes, and for a 1920×1080 resolution video (8100 16×16 macroblocks), the laminar flow rate is R lam= 12×8100×30 = 2.916Mbps. After the turbulence parameters are quantized and encoded in 8 dimensions, each macroblock has an average of 112 bytes, and the turbulence code rate is R turb = 112×8100×30 = 27.216Mbps. Total bit rate R total = R lam + R turb = 30.132Mbps.

[0060] The bitrate ratio of the laminar flow field and the turbulent flow field is dynamically allocated according to the preset target bitrate and texture complexity; the texture complexity is determined by the energy proportion of the turbulent field.

[0061] In this embodiment, the target bit rate R is set t = 20Mbps, requiring 33.6% compression. Texture complexity is determined by calculating the energy proportion of the turbulence field: α = E turb / (E turb + E lam ), where energy E =Σ|v| 2 For the flame scene, the measured α is 0.73, indicating that turbulent motion is dominant. The code rate allocation strategy is: the adjusted laminar code rate R' lam = R t ×(1-α) =5.4Mbps, the adjusted turbulence bit rate R' turb = R t ×α = 14.6Mbps.

[0062] Based on the dynamic allocation result, the actual bit rate is obtained. When the actual bit rate exceeds the target bit rate, the quantization step size of the turbulence parameters is adjusted to maintain the coding accuracy of the laminar motion parameters, and the updated bit rate allocation result is obtained.

[0063] In this embodiment, the quantization step size of the turbulence parameter is changed from the default Δ = (h max -h min ) / 255 is adjusted to Δ'= k×Δ, where k = sqrt(R' turb / R turb ) = 0.73. Specifically, the quantization level is reduced from 256 to 187, and the quantization formula is modified to h q = round((h - h min ) / (k×(h max -h min ))×186). Laminar motion parameters maintain 16-bit precision to ensure overall motion accuracy.

[0064] The bitrate allocation is iteratively optimized through a closed-loop feedback mechanism so that the output bitstream meets the target bitrate constraint.

[0065] In this embodiment, the actual bit rate R is evaluated every 10 frames. actual and target bit rate R t The deviation of bit rate control error is defined as e = (R actual - R t ) / R t When |e| > 0.05, an adjustment is triggered: if e > 0, the quantization step size is increased by k' = k×(1 + 0.1×e); if e < 0, the quantization step size is reduced by k' = k×(1 - 0.1×|e|). The reconstruction quality is also monitored. When VMAF falls below 85, the maximum value of k is limited to 1.5.

[0066] For example, in some embodiments, the target bitrate is set to 15 Mbps to adapt to the network bandwidth. The initial encoding produces a bitrate of 18.7 Mbps, which exceeds 24.7%. The system detects that the flame area α = 0.81 and expands the turbulence quantization step by 1.25 times. After three iterative adjustments, the actual bitrate stabilizes in the range of 14.8-15.2 Mbps. There is no significant decline in the audience's subjective experience. Optionally, the bitrate control introduces scene switching detection. When the texture complexity α of three consecutive frames changes by more than 0.3, it is determined to be a scene switch and the bitrate allocation parameters are reset immediately. It is particularly effective when switching from a static scene to an explosion scene, avoiding a brief drop in quality. Through the above steps, this embodiment achieves precise bitrate control. Test data shows that under the target bitrate constraint, the bitrate fluctuation is reduced, while the average VMAF only decreases slightly, maintaining visual quality.

[0067] According to one aspect of the present application, an adaptive parameter adjustment mechanism based on scene type is also included:

[0068] The texture complexity and motion pattern features of the input video frame are extracted to construct a feature vector, and the scene type is determined based on the feature vector; the scene type includes flames, flowing water, smoke or clouds.

[0069] In this embodiment, the texture complexity is evaluated by calculating the discrete cosine transform (DCT) coefficient distribution of an 8×8 block. The complexity index C is defined as tex = Σ|DCT coef (u, v)| / (u+v+1), where (u, v) is the frequency coordinate, DCT coef is the DCT coefficient. The motion pattern features include: motion vector amplitude histogram H mv (32 quantization levels), directional distribution D of motion field gradient ▽v grad (8 directions), temporal motion consistency index T cons = |v t -vt-1 |. Eigenvector f = [C tex , H mv , D grad , T cons ]∈R 43 A pre-trained random forest classifier was used, which contained 100 decision trees with a maximum depth of 15. The training set contained 5,000 samples of each of four scene types: flame (label 0), flowing water (label 1), smoke (label 2), and fog (label 3). The classifier outputs a probability vector p = [p0, p1, p2, p3]. When max(p) > 0.7, the scene type is determined, otherwise it is classified as a mixed scene. The flame scene is characterized by high texture complexity (C tex > 120) and vertical motion dominant (D grad centered at 90°±30°).

[0070] Coding parameter templates are preset for different scene types, including fluid decomposition threshold, neural network quantization step size and laminar turbulence bit rate allocation ratio.

[0071] In this embodiment, the parameter templates are stored in a lookup table: Flame template: Fluid decomposition energy threshold τ fire =0.15, quantization step size Q fire = 1.2, code rate allocation ratio r fire = 0.25:0.75; pipeline template: τ water = 0.08, Q water = 0.8, r water = 0.4:0.6; smoke template: τ smoke = 0.12, Q smoke = 1.0, r smoke = 0.3:0.7; cloud template: τ cloud = 0.05, Q cloud = 0.6, r cloud = 0.5:0.5.

[0072] The corresponding coding parameter template is selected according to the determined scene type, and the energy threshold of the fluid dynamics decomposition, the quantization accuracy of the neural ordinary differential equation network, and the allocation strategy of the bit rate control are dynamically adjusted.

[0073] In this embodiment, flame template parameters are loaded immediately after a flame scene is detected. During fluid decomposition, the energy threshold for the Helmholtz-Hodge iteration is set to 0.15, allowing more high-frequency components to be attributed to the turbulent field. The quantization step size of the neural ODE network's potential representation is increased by a factor of 1.2 to tolerate greater quantization error and reduce the bitrate. The bitrate split between laminar flow and turbulent flow is adjusted to 25%:75% because the random motion of the flame dominates.

[0074] For example, in some embodiments, the system detects that the scene switches from a forest background (cloud type) to a campfire close-up (flame type) in frame 127. The classifier output p = [0.89, 0.03, 0.06, 0.02], confirming that it is a flame scene. The encoder smoothly switches the parameter within the 2-frame transition period: τ gradually changes from 0.05 to 0.15 to avoid mutation artifacts. After the switch, the proportion of turbulent field energy in the flame area is increased, the flame jump is captured more accurately, and the bit rate and VMAF score are improved. In some embodiments, the mixed scene adopts a parameter interpolation strategy. When the waterfall scene contains both water flow and water mist, p = [0.1, 0.52, 0.05, 0.33], and the system calculates the weighted parameter: τ mix = Σp i ×τ i = 0.074, achieving smooth transition. Preferably, the parameter template supports online update. The system records the encoding effect of each scene, and after accumulating 100 GOPs, fine-tunes the template parameters through Bayesian optimization. fire The average VMAF score was improved from an initial 0.15 to 0.17. Through these steps, scene-adaptive intelligent encoding was achieved. On a test set containing a variety of dynamic textures, average encoding efficiency was improved and the rate of incorrect scene determinations was reduced.

[0075] According to one aspect of the present application, encoding is also performed using a temporal multi-scale prediction coding structure:

[0076] A continuous sequence of input video frames is organized into a hierarchical temporal structure consisting of key frames and bidirectionally predicted frames.

[0077] In this embodiment, a four-layer temporal hierarchy is employed. Layer 0 consists of key frames (I frames), spaced every 90 frames; layer 1 consists of forward-predicted frames (P frames), spaced every 30 frames; layers 2 and 3 consist of bidirectionally predicted frames (B frames), spaced every 15 and 5 frames, respectively. The frame types are arranged in the following pattern: I-B3-B3-B2-B3-B3-P1-..., where the subscripts indicate the level. This structure forms a pyramid in the temporal dimension, supporting scalable decoding.

[0078] Perform the complete encoding process from motion field extraction to bitstream multiplexing for key frames.

[0079] In this embodiment, the key frame I0 performs complete motion field extraction, fluid decomposition, laminar affine encoding and turbulent ODE encoding. Specifically, the laminar flow parameters M of I0 are I = {t x , t y ,θ,s, k x , k y} completely preserved, turbulence potential representation h I ∈R 8 ×120×68 After full 8-bit quantization, the key frame data size is approximately 2.8MB, including all the information required for reconstruction.

[0080] For the bidirectional prediction frame, only the time increment of its motion field relative to the preset reference frame is encoded; the turbulence field change is predicted through the time evolution characteristics of the neural ordinary differential equation network, and the prediction residual is encoded.

[0081] In this embodiment, for a P frame P 30 , calculate its motion field v P With reference I frame motion field v I The difference: Δv = v P -v I Laminar flow increment ΔM P = M P -M I Usually small, further compressed using entropy coding, where M P is the set of laminar motion parameters of the predicted frame, M I is the set of laminar motion parameters of the reference keyframe. The turbulence increment is predicted by neural ODE: h P pred =ODESolve(h I , t=1.0s), only the prediction residual r needs to be encoded P = h P -h P pred To reduce the dynamic range of the residual, 4-bit quantization can be used. B frames use bidirectional time information. 45 (Located between I0 and P 60 Between), through ODE bidirectional prediction: forward h 45 fwd =ODESolve(h0,t=1.5s), backward h 45 bwd = ODESolve(h 60 , t=-0.5s). Fusion prediction h 45 pred = 0.7 × h 45 fwd + 0.3×h45 bwd , the weight is determined according to the time distance. Only the extremely small residual r is encoded 45 = h 45 -h 45 pred , the data volume is reduced to 0.3MB / frame. ODESolve is a neural network ordinary differential equation solver, h P is the actual turbulence latent representation of the predicted frame.

[0082] The complete coding parameters of the key frames and the time increment parameters of the bidirectional prediction frames are combined to form an output bitstream with temporal scalability.

[0083] In this embodiment, the bitstream is organized into a layered structure: the base layer contains all I-frames and P-frames, with a bitrate of 6.2 Mbps, capable of independent decoding to produce 10 fps video. Enhancement layer 1 adds a second layer of B-frames, with a total bitrate of 9.8 Mbps, increasing the frame rate to 20 fps. Enhancement layer 2 contains all B-frames, requiring 13.5 Mbps for a full 30 fps. Each layer header indicates layer information and dependencies.

[0084] For example, in some embodiments, a GOP length of 90 frames covers 3 seconds. The moment of skill release is set as a key frame to fully record the explosion texture. Subsequent P frames capture the overall trend of flame spread, and the laminar flow increment shows the expansion movement (s increases from 1.0 to 1.8). The B frame predicts the details of the flame swing through ODE, and the predicted PSNR of frame 45 reaches 41.2dB, and only 3.2% of the residual information needs to be encoded. When the network is congested, the client can choose to decode only the base layer and still maintain the dynamics of the flame body. In another embodiment, the key frame insertion adopts an adaptive strategy. Monitor the accumulation of ODE prediction errors, and when the turbulent residual energy E r = Σ|r| 2 > 0.1×E h When E h is the total energy potentially represented by turbulence. This is particularly effective when flames suddenly change color or intensity. Through the above steps, efficient encoding in the temporal dimension is achieved. Tests have shown that the multi-scale structure improves VMAF scores compared to a fixed GOP at the same bitrate. It also supports three smooth switching speeds of 10, 20, and 30 fps to adapt to varying network conditions.

[0085] According to one aspect of the present application, extracting a pixel-level motion field of an input video frame includes:

[0086] Construct a multi-scale feature pyramid of the input video frame and its adjacent frames, and iteratively estimate the optical flow field at each scale level from coarse to fine.

[0087] In this example, a five-layer pyramid is constructed for three consecutive frames, with the resolution halved at each layer. The original 1920×1080 resolution is gradually downsampled to 120×68. The optical flow at the coarse scale is initialized to zero and then upsampled using bilinear interpolation to provide an initial estimate for the fine scale. Each layer is iterated five times to optimize the optical flow field, transferring motion information from coarse to fine scales.

[0088] The bidirectional optical flow consistency constraint is adopted to calculate the symmetric error of the forward optical flow and the backward optical flow in the optical flow field, and the unreliable motion estimation in the occluded area is eliminated to obtain the optimized optical flow field.

[0089] In this embodiment, the forward optical flow v is calculated fwd and backward optical flow v bwd , check consistency: |v fwd + warp(v bwd Regions with a )| < 0.5 pixels are marked as reliable, otherwise they are marked as occluded. The occlusion binary image (mask) occupies approximately 8-12% of the frame, primarily located at motion boundaries. For occluded regions, motion estimates are interpolated from neighboring reliable regions.

[0090] Anisotropic diffusion filtering is used to impose smoothness constraints on the optimized optical flow field, which can suppress noise in texture areas while maintaining motion boundaries and generate a continuous and smooth pixel-level motion field.

[0091] In this embodiment, the diffusion coefficient is adaptively adjusted according to the image gradient: c(▽I) = exp(-|▽I| 2 / κ 2 ), where κ = 20 controls the edge preservation strength. c → 0 at motion boundaries suppresses diffusion, while c → 1 in textured regions enhances smoothing. After 10 iterations, motion field noise is reduced by 65% ​​while maintaining edge sharpness.

[0092] According to one aspect of the present application, generating laminar flow motion parameters comprises:

[0093] The local motion consistency of the laminar flow field is analyzed, and the coding blocks are adaptively divided according to the spatial correlation of the motion vectors, so that the motion within the block remains highly consistent.

[0094] In this embodiment, starting from a 16×16 basic block, the intra-block motion variance σ is calculated 2 v When σ 2 v < 2.0, merge into 32×32 blocks; when σ 2 v When the value is greater than 8.0, the image is split into 8×8 blocks. The flame edge uses 8×8 fine blocks, and the flame body uses 32×32 large blocks, with an average compression of 40% of the number of blocks.

[0095] An affine motion model with at least six parameters is fitted to each coding block to obtain a set of block motion parameters; the six parameters include horizontal translation, vertical translation, rotation, scaling, and horizontal and vertical shear transformations; based on the block motion parameter set, the affine parameters of each coding block are optimized by minimizing the difference in motion vectors at the boundaries of adjacent coding blocks, maintaining the spatial continuity of the motion field, and generating laminar motion parameters.

[0096] In this embodiment, the affine transformation is expressed as: [x'] = [ab][x] + [e], [y'] = [cd][y] + [f]. The six parameters [a, b, c, d, e, f] are obtained by least squares fitting. The parameter difference between adjacent blocks |Δa|+|Δb|+... is controlled within 0.05 to ensure motion continuity. Typical parameters of the flame scene: translation (e, f) = (3.2, -1.5), rotation and scaling matrix main diagonal elements a≈d≈1.02. Where x, y are the coordinates of the original pixel points; x', y' are the coordinates of the transformed pixel points; a, d are the main diagonal elements of scaling and rotation; b, c are the non-diagonal elements of rotation and shear transformation; e and f are translation parameters, representing the translation amounts in the horizontal and vertical directions, respectively.

[0097] For example, in some embodiments, the main flow region exhibits consistent motion, requiring only 200 affine parameter sets for encoding using large 32×32 blocks. The splash region exhibits complex motion and is automatically subdivided into 8×8 blocks. The average error between the motion field predicted by the affine parameters and the true optical flow is 0.7 pixels, meeting the visual losslessness requirement. Through these steps, the laminar flow field is compressed to 2.4KB / frame, significantly improving the compression ratio.

[0098] According to one aspect of the present application, before performing motion field extraction, it also includes adaptive selection of a coding mode:

[0099] The motion vector magnitude distribution and texture direction gradient of the input video frame are calculated to analyze the motion intensity and texture complexity of the video frame.

[0100] In this embodiment, the motion vector amplitude histogram is statistically analyzed to calculate the average value μ v and standard deviation σ v The texture gradient is extracted by Sobel operator and the gradient direction histogram is statistically calculated. The motion intensity index M = μ is defined v / σ v , texture complexity T = H(grad dir ), where H is the entropy function, grad dir is the gradient direction of each pixel in the image.

[0101] According to the joint characteristics of motion intensity and texture complexity, the coding mode is adaptively selected: when the motion is smooth and the texture is simple, the laminar-only coding mode is adopted; when there is complex texture but weak motion, the turbulence-only coding mode is adopted; when both motion and texture are complex, the full fluid dynamics decomposition mode is adopted; based on the selected coding mode, the corresponding encoding steps in the method are selectively executed, skipping unnecessary processing to reduce computational complexity.

[0102] In this embodiment, a mode selection rule is established: when M < 2.0 and T < 3.5, the motion is smooth and the texture is simple, so the laminar flow only mode is selected; when M < 1.0 and T > 5.0, the motion is weak and the texture is complex, so the turbulence only mode is selected; in other cases, the complete fluid decomposition mode is selected. Each mode has a preset dedicated processing path. For example, in some embodiments, M = 0.8 and T = 6.2 triggers the turbulence only mode. Motion estimation and laminar coding are skipped, and ODE modeling of the texture is directly performed, which reduces encoding time and bit rate. Through mode adaptation, the overall encoding complexity is reduced while keeping the quality loss within 0.5dB.

[0103] According to one aspect of the present application, predicting a continuous evolution trajectory of a potential representation in the time domain includes:

[0104] In the time step Δt, the dynamic function f θ Calculate four intermediate slopes: k1 is based on the current state, k2 and k3 are based on the midpoint predicted state, and k4 is based on the end point predicted state; combine the four intermediate slopes with a weight of 1:2:2:1 and update the potential representation h(t+Δt) = h(t) + Δt(k1+2k2+2k3+k4) / 6.

[0105] In this embodiment, given the current state h(t) and the time step Δt, four slopes are calculated: k1 = f θ (h, t), k2 = f θ (h + 0.5Δt·k1, t + 0.5Δt), k3 = f θ (h + 0.5Δt·k2, t + 0.5Δt), k4 = f θ (h + Δt k3, t + Δt). The updated formula h(t+Δt) = h(t) + (Δt / 6)(k1 + 2k2 + 2k3 + k4) provides fourth-order accuracy.

[0106] Through the adaptive step size control mechanism, the time step size Δt is dynamically adjusted according to the local error estimation, optimizing the computational efficiency while ensuring the numerical accuracy, and generating a smooth and continuous time domain evolution trajectory.

[0107] In this embodiment, the estimated local truncation error ε = |h RK4 -h RK2 |, when ε > 10 -4 The step size is halved when ε<10 -5 The step size is doubled when the flame is changing rapidly. The step size is limited to the range of [0.001, 0.1] seconds. The step size is automatically reduced to 0.003 seconds when the flame is changing rapidly and increased to 0.05 seconds when the flame is evolving steadily. RK4 is the potential representation after time step Δt obtained using the fourth-order Runge-Kutta method (RK4), h RK2 is the state estimate at the same time t+Δt calculated using the second-order Runge-Kutta method (RK2).

[0108] According to one aspect of the present application, the Helmholtz decomposition adopts an adaptive threshold determination mechanism based on energy spectrum:

[0109] A two-dimensional fast Fourier transform is performed on the pixel-level motion field to calculate the frequency domain energy spectrum distribution. Combined with the frequency domain energy spectrum distribution, the cumulative energy ratio is calculated from low to high frequency, and the frequency boundary between laminar and turbulent flow is determined by detecting the inflection point of the cumulative energy curve, where the low-frequency component corresponds to laminar motion and the high-frequency component corresponds to turbulent motion.

[0110] In this embodiment, after two-dimensional fast Fourier transform (FFT), the energy spectrum E(f x , f y ) = |F(v)| 2 . According to the radial frequency f r = sqrt(f x 2 + f y 2 ) Statistical energy distribution. Calculate the cumulative energy percentage. When it reaches 85%, the corresponding frequency f c As the laminar / turbulent boundary. Flame scene f c ≈15Hz, water flow scene f c ≈8Hz. Where f x , f y are the horizontal and vertical frequency components in the two-dimensional Fourier transform; F(v) is the frequency domain representation obtained by performing two-dimensional fast Fourier transform on the pixel-level motion vector field v(x, y).

[0111] The iterative convergence threshold of the Helmholtz decomposition is dynamically set according to the frequency boundary, so that the low-frequency dominant region is preferentially decomposed into potential flow components, and the high-frequency dominant region is preferentially decomposed into vortex components, thereby achieving adaptive and accurate decomposition of the motion field.

[0112] In this embodiment, below f cThe components are preferentially distributed to the laminar flow field, which is higher than f c Assigned to the turbulent field. The convergence threshold of the decomposition iteration τ* = 0.1 / f c , achieving a finer decomposition of the high-frequency region and reducing the decomposition error. Through the above steps, the cumulative error of the ODE prediction of 5 frames is reduced, and the decomposition accuracy guided by the energy spectrum is improved.

[0113] According to one aspect of the present application, a physical constraint loss function is used when training a neural ordinary differential equation network:

[0114] Based on the reconstruction loss, the fluid physics constraint term is introduced:

[0115] Calculate and predict the divergence of the motion field, ensuring mass conservation by minimizing divergence loss; calculate the time derivative of vorticity, maintaining the rotational motion characteristics through vorticity conservation loss; calculate the change in kinetic energy, and constrain the total energy of the system through energy conservation loss;

[0116] The reconstruction loss, divergence loss, vorticity loss and energy conservation loss are adaptively weighted and combined, where the weight coefficients are dynamically adjusted according to the relative importance of different physical quantities and the convergence state;

[0117] Through the joint optimization of the physical constraint loss function, the neural network can follow the basic laws of fluid dynamics while learning the time domain evolution law.

[0118] According to one aspect of the present application, the adjoint sensitivity method is used to calculate the parameter gradient when training a neural ordinary differential equation network:

[0119] Starting from the loss gradient at the end time, the adjoint equation is constructed and reversely solved using a fourth-order Runge-Kutta integrator to obtain the value of the adjoint variable at each time;

[0120] During the reverse integration process, the dynamics function f is calculated synchronously θ Local gradients with respect to network parameters are accumulated internally with adjoint variables to avoid storing the full forward trajectory;

[0121] By obtaining the gradients of all parameters simultaneously through a single backward integration, the memory consumption is reduced compared to the traditional back-propagation method, making efficient training of deep neural ODE networks possible.

[0122] In one embodiment of the present application, the total loss L = L recon + 0.1L div + 0.05L vort +0.02L energy . Reconstruction loss L recon = MSE(v, v'). Divergence loss L div = |▽·v'|2 Ensure mass conservation. Vorticity loss L vort = |Ξω / Ξt - v*▽ 2 ω| 2 Maintain the rotation characteristics. Energy loss L energy = |E(t) - E(0)| to constrain the total energy change. The adjoint equation da / dt = -Ξf / Ξh·a - ΞL / Ξh is constructed and integrated backward from the endpoint. The parameter gradient dL / dθ = ∫a·Ξf / Ξθ dt is simultaneously accumulated. A single backpropagation obtains all gradients, reducing memory requirements from O(NT) to O(N), where N is the number of parameters, T is the number of time steps, ω is the vorticity field, E(t) is the total energy at time t, a is the adjoint variable, and f is the dynamics function. When training ODE networks, physical constraints reduce vorticity preservation errors. The adjoint method enables the training of ODE networks with a depth of 100 layers, improving training efficiency.

[0123] According to one aspect of the present application, a spatiotemporal joint optimization is performed on the pixel-level motion field:

[0124] A spatiotemporal joint energy function is constructed, which includes temporal and spatial terms. The temporal term ensures the temporal continuity of the motion trajectory by minimizing the acceleration of the motion field in adjacent frames. The spatial term maintains the spatial coherence of the motion field through edge-preserving smoothness constraints.

[0125] The alternating direction multiplier method is used to separately optimize the temporal and spatial constraints, optimizing temporal continuity when the spatial smoothness is fixed, and optimizing spatial smoothness when the temporal trajectory is fixed.

[0126] By iteratively solving the spatiotemporal joint optimization problem, a high-quality motion field is generated that is continuous in time, smooth in space, and maintains clear motion boundaries.

[0127] According to one aspect of the present application, the temporal multi-scale predictive coding structure adopts a content-based adaptive GOP configuration:

[0128] Analyze the motion complexity and turbulence intensity changes of continuous frame sequences, and evaluate the prediction coding efficiency by calculating the laminar / turbulent energy ratio;

[0129] When a scene switch is detected or the turbulence intensity exceeds a preset threshold, the current GOP is terminated and a new keyframe is inserted; when the laminar energy ratio of the motion field exceeds the set ratio, the GOP length is extended to the maximum allowable value;

[0130] The level depth of the prediction frame in the GOP is dynamically adjusted according to the motion characteristics. A shallower B-frame reference level is set for the segment dominated by turbulent energy, and a deeper B-frame reference level is set for the segment dominated by laminar energy.

[0131] In one embodiment of the present application, a spatiotemporal joint energy function is constructed to optimize the motion field. Energy function E = E temporal + E spatial The time domain term E temporal = ∫|Ξ 2 v / Ξt 2 | 2 dt minimizes acceleration. Spatial term E spatial = ∫c(▽I)|▽v| 2 dx maintains spatial smoothness. Alternating optimization is used: fix the spatial domain to solve the time domain, then fix the time domain to solve the spatial domain, and iterate until convergence. Calculate the laminar / turbulent energy ratio R = E lam / E turb . When R > 2.0, laminar flow dominates and the GOP is extended to 120 frames; when R < 0.5, turbulent flow dominates and the GOP is shortened to 60 frames. Scene switch detection: an I frame is inserted immediately when the R change of 3 consecutive frames exceeds 100%. The B frame level is adjusted according to the turbulence intensity: 4 layers are used for weak turbulence, and 2 layers are limited to strong turbulence. For example, in some embodiments, R = 2.5 at the beginning of the fire, and a long GOP is used; after the fire spreads, R = 0.3, the GOP is automatically shortened and the B frame level is reduced. The coding efficiency is improved while avoiding the accumulation of prediction errors. Comprehensive tests show that compared with the H.265 standard, the bit rate of the dynamic texture area is reduced and the VMAF score is improved.

[0132] According to another aspect of the present application, a dynamic texture video coding method is based on the fusion of mathematical abstraction of fluid dynamics equations and neural networks, specifically comprising: extracting the dual-channel motion field of the video frame through the optical flow estimation module, and separating it into a laminar flow field (irrotational component) and a turbulent flow field (rotational component) using the Helmholtz decomposition theorem, wherein the laminar flow field satisfies ▽×v lam =0 constraint, the turbulent field satisfies ▽·v turb = 0. For turbulent field modeling, a neural ODE network is designed to solve the simplified form of the Navier-Stokes equation: Ξv / Ξt=v*▽ 2 v-(v·▽)v, where v* is the dynamic viscosity coefficient (range 0.01-0.05). The network uses the fourth-order Runge-Kutta method for numerical integration, and the time step Δt is set to 1 / 30 second. The texture statistical characteristics are maintained by Gram matrix matching: G(I) ij =∑ k F(I) ik F(I) jk, where F(I) is the relu3_3 layer feature map of the VGG-19 network. The encoder only transmits the laminar flow field motion vector, turbulence field parameters (including 8-dimensional latent code and 3-dimensional vorticity coefficient) and texture primitives of the key frame. The decoder reconstructs the dynamic texture through the particle system. The particle motion follows the improved smoothed particle hydrodynamics (SPH) equation: i =∑ j m j (p i +p j ) / (ρ i ρ j ) ▽W(r ij , h), where the smoothing length h is set to 4 times the pixel spacing. i is the acceleration vector of the i-th particle; m j is the mass of the jth neighboring particle; p i is the pressure of the i-th particle, ρ i is the density of the i-th particle, W( ) represents the kernel function, r ij is the distance vector between the i-th and j-th particles.

[0133] This implementation consists of six core stages: the motion field decomposition stage, which uses the Farneback optical flow algorithm to calculate the initial motion field and Hodge decomposition to obtain the laminar and turbulent components; the parameter extraction stage, which performs affine motion modeling and encoding on the laminar field, and reduces the turbulent field to a latent space using an 8-layer convolutional autoencoder; the time-domain prediction stage, in which a neural ODE network uses the turbulence parameters of the current frame as initial conditions to predict the vorticity field evolution for the next five frames; the texture synthesis stage, in which the decoder drives the particle system based on the motion parameters, maintaining a 1:4 mapping between particle density and input video resolution; the detail enhancement stage, which uses the pre-trained StyleGAN2 network to refine the local texture of the reconstructed area; and the quality assessment stage, which online calculates the difference between the Gram matrix of the reconstructed frame and the original frame and dynamically adjusts the quantization parameter (QP) of the next GOP. The entire process forms a closed-loop optimization system.

[0134] Taking forest fire monitoring video encoding as an example, the specific implementation process is as follows: After receiving 1080p @ 30fps input video, the system first detects dynamic texture areas (macroblocks with flames accounting for more than 15% of the area). For macroblocks marked as texture areas, the optical flow module calculates the motion vector for each pixel, decomposing the laminar flow field (reflecting the overall spread of the flame) and the turbulent flow field (characterizing the random motion of sparks). The turbulence parameter encoder compresses this into a compact 24-byte representation for each macroblock and also records the flame color primitives (YUV values) of the keyframes. After receiving the data stream, the particle system determines the direction of the main flame movement based on the laminar flow parameters (updated once per second). The turbulence parameters control the Brownian motion intensity of 5,000 particles, and the flame effects are rendered in real time using OpenGL shaders. If the flame area suddenly expands by 30%, the system automatically inserts a keyframe and resets the ODE network state to ensure reconstruction quality in sudden changes.

[0135] This embodiment introduces Helmholtz decomposition to decompose the video frame into laminar flow field and turbulent pulsation field, where the laminar flow field is coded using HEVC standard motion compensation, and the turbulent field is modeled by a neural ODE network: du' / dt = f θ (u', ​​t), θ is a learnable parameter, u' is the turbulent pulsation field. Design the frequency domain loss function L based on the Gram matrix gram =||Gram(FFT(u'))-Gram(FFT(u))|| 2 , ensuring that the turbulent energy spectrum characteristics remain stable during the compression process.

[0136] In a specific embodiment of the present application, a dynamic texture video encoding method includes the following steps:

[0137] Step 1: Video input and motion field decomposition This system receives the input video stream in YUV420 format and first extracts the luminance component Y for motion analysis. The improved Farneback dense optical flow algorithm is used to calculate the pixel-level motion field v(x, y, t), and its mathematical model is: v = argmin v ∑ W |I t +▽I·v| 2 +λ|▽v| 2 , where I t is the brightness component of the t-th frame image, W is the 15×15 search window, and λ=0.1 controls the weight of the smoothing term. The Helmholtz decomposition theorem is then applied to separate the motion field into laminar components v lam and the turbulent component v turb , satisfying: ▽×v lam =0, ▽·v turb= 0, the decomposition process is achieved by 50 iterations of the Hodge projection algorithm, and the single frame processing time is optimized to 12ms (1080p resolution). After separation, the laminar flow field is encoded as the affine motion model parameters m affine ∈R6, the turbulent field enters the subsequent processing module.

[0138] Step 2: Extracting turbulence field features and compressing turbulence field v turb After 8 layers of convolutional autoencoder dimensionality reduction, the network structure is: h=Encoder θ (v turb )=f conv8 (f conv7 (...f conv1 (v turb ), where each layer has a convolution kernel size of 3×3, a number of channels of [64, 128, 256, 512, 512, 256, 128, 8], and outputs a latent code h∈R 8×H / 16×W / 16 The quantization module uses uniform 8-bit quantization: h*=round((hh min ) / (h max -h min )×255). Tests show that this encoder compresses turbulent flow field data to 0.32bpp in a flame scene, maintaining a PSNR above 42dB. The compressed parameter stream is multiplexed with the laminar motion vectors to form the transmission bitstream.

[0139] Step 3: Neural ODE time domain prediction At the decoding end, the neural ODE network predicts the evolution of the turbulent field, and the dynamic equation is defined as: (dh(t)) / dt=f θ (h(t), t), f θ :R 8 →R 8 , where f θ It is a 5-layer MLP (256-512-512-256-8), and the activation function is Softplus: Softplus(x)=ln(1+e x ) to ensure second-order differentiability. The fourth-order Runge-Kutta method is used to numerically solve: h t+Δt =h t +(Δt / 6)(k1+2k2+2k3+k4), with the constraints of a Lipschitz constant L < 1.5 (implemented by weighted spectral normalization) and a time step of Δt = 0.2 frame intervals. This module reduces the long-term prediction error accumulation rate to 0.3% per frame and supports a GOP length of 90 frames.

[0140] Step 4: Particle system texture reconstruction Based on the predicted turbulence parameter h*, the decoder uses the improved SPH particle system to reconstruct the dynamic texture. Particle motion equation: xi t+1 = x i t + ∑ j∈N(i) (m j (p i + p j )) / (ρ i ρ j ) ▽W(|x i - x j |, h), where the smoothing kernel W(r, h) = 315 / (64πh 9 )(h 2 - r 2 ) 3 (r < h), the kernel radius h = 4Δx, and Δx is the pixel pitch. The system is configured with 5000 particles / macroblock and parallel updates are implemented through an OpenGL compute shader, with a rendering speed of 8 ms / frame (RTX4090 GPU).

[0141] Step Five: Gram Matrix Quality Constraint To maintain texture statistical properties, define the Gram matrix loss based on the VGG-19 feature map: L gram = 1 / (C 2 HW) ∑ i=1 C ∑ j=1 C ( G ij orig - G ij rec ) 2 , where G ij = ∑ k=1 HW F ik F jk , F ∈ R C×HW is the feature of the relu3_3 layer (C = 256). This loss term is combined with the MSE loss with a weight ratio of 0.7:0.3, and the decoder parameters are adjusted through the Adam optimizer (lr = 3×10 -4 ) to make the energy spectrum error Δμ of the reconstructed texture < 0.03. Where G ij orig [[ID=​​​​​​​​​+α(VMAF target -VMAF current ), where the proportional coefficient α = 0.5, the quantization parameter (controls the rate-distortion trade-off) QP reference value QP base =32. The final bitstream structure includes: header information (2 bytes): GOP length, resolution, etc.; laminar flow field MV (6×4 bytes): affine motion parameters; turbulence parameters (8×N macroblocks): 8-bit quantization value. target Score target video quality, VMAF current The actual VMAF score for the current frame or current GOP.

[0143] In terms of compression efficiency, this embodiment reduces the bitrate in turbulent regions through parametric representation driven by a physical model. In terms of reconstruction quality, it improves the VMAF score, particularly the temporal coherence of texture details. The neural ODE network requires only a five-layer MLP structure, which improves inference speed. Parametric modeling of turbulence extends the I-frame interval and reduces keyframe overhead. Gram matrix constraints reduce texture statistical errors and effectively avoid mode collapse in generated textures. This resolves the conflict between fidelity and computational cost in dynamic texture coding. By integrating fluid dynamics and deep learning modeling, a theoretical framework for dynamic texture coding is reconstructed.

[0144] This invention can be applied to streaming video services, improving the transmission efficiency and visual quality of highly dynamic natural scene videos and addressing key technical pain points for OTT platforms. For dynamic textures such as waterfalls and flames in 4K HDR nature documentaries, traditional encoders require a bitrate of 18-25 Mbps to maintain a VMAF score of 80. However, this invention uses laminar / turbulent field decomposition technology to reduce the bitrate in dynamic texture areas while simultaneously improving the VMAF score. It can also be applied to virtual reality and the metaverse, providing a real-time, physically accurate dynamic texture encoding solution for cloud VR scenes, breaking through the transmission bottleneck of interactive content. In 6DoF virtual water flow interaction scenarios, traditional solutions suffer from asynchronous haptic feedback due to encoding delays exceeding 200ms. This invention uses a neural ODE network to predict texture evolution, compressing end-to-end latency. It can also be applied to intelligent traffic monitoring, effectively improving the reliability of video analysis in inclement weather and addressing perception challenges in autonomous driving systems. For rainy scenes in road condition datasets, traditional H.265 encoding distorts raindrop textures, reducing YOLOv7 object detection mean average average performance (MAP). This invention uses optical flow-guided field separation technology to decouple vehicle rigid motion (laminar flow) from rain and snow turbulence, enabling parametric modeling of rainfall patterns. A 3D convolutional network is used to extract the rainline directional field; a turbulence parameter encoder generates a 12-dimensional feature vector; and raindrops reconstructed at the decoder conform to a Pareto distribution (shape parameter α = 1.2 ± 0.05). When deployed on relevant hardware, compared to H.265 High Tier, this method improves object detection mean average approach (mAP) while reducing video storage requirements and minimizing raindrop motion vector prediction errors.

[0145] The preferred embodiments of the present invention are described in detail above. However, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all fall within the scope of protection of the present invention.

Claims

1. A dynamic texture video encoding method, characterized in that: include: Obtain an input video frame and extract a pixel-level motion field of the input video frame; Perform fluid dynamics decomposition on the pixel-level motion field to obtain a laminar flow field that satisfies the irrotation constraint and a turbulent flow field that satisfies the divergence constraint; Encode the laminar flow field with an affine motion model to generate laminar flow motion parameters; The turbulence field is input into the pre-built neural ordinary differential equation network for parameterized encoding to generate turbulence parameters; Multiplexing laminar flow motion parameters and turbulence parameters into output code stream; Generate turbulence parameters, including: Input the turbulence field into the convolutional encoder to extract the potential representation of the turbulence field; The continuous evolution trajectory of the potential representation in the time domain is predicted through the neural ordinary differential equation network to generate turbulence parameters; Extracting a latent representation of the turbulence field, including: Construct an 8-layer convolutional neural network and map the turbulence field into a high-dimensional feature space through layer-by-layer feature extraction; Compress the features in the high-dimensional feature space into an 8-dimensional latent code space through progressive dimensionality reduction, generating a latent code of size 8×H / 16×W / 16, where H and W are the height and width of the input video frame respectively; Perform 8-bit uniform quantization on the latent code to generate a quantized latent representation; Multiplexing laminar motion parameters and turbulence parameters into output bitstream includes closed-loop rate control mechanism: The code rates of laminar motion parameters and turbulence parameters are evaluated separately; Dynamically allocate the bitrate ratio of laminar flow field and turbulent flow field according to the preset target bitrate and texture complexity; the texture complexity is determined by the energy proportion of the turbulent field; Based on the dynamic allocation result, the actual bit rate is obtained. When the actual bit rate exceeds the target bit rate, the quantization step size of the turbulence parameter is adjusted to obtain the updated bit rate allocation result; Iteratively optimize the bitrate allocation through a closed-loop feedback mechanism to ensure that the output bitstream meets the target bitrate constraint; Generate laminar flow motion parameters, including: Analyze the local motion consistency of the laminar flow field and adaptively divide the coding blocks according to the spatial correlation of the motion vector; Fitting each coding block with an affine motion model of at least 6 parameters to obtain a set of block motion parameters; the 6 parameters include horizontal translation, vertical translation, rotation, scaling, and horizontal and vertical shear transformations; Based on the block motion parameter set, the affine parameters of each coding block are optimized by minimizing the difference in motion vectors at the boundaries of adjacent coding blocks to generate laminar motion parameters.

2. The method according to claim 1, characterized in that The laminar and turbulent fields are obtained, including: Apply the Helmholtz decomposition theorem to the pixel-level motion field and decompose the motion field into potential flow component and vortex component; The Hodge projection algorithm is used to iteratively optimize the potential flow component and the vortex component, so that the potential flow component satisfies the irrotation constraint and the vortex component satisfies the divergence constraint. The potential flow component that satisfies the irrotational constraint is regarded as the laminar flow field, and the vortex component that satisfies the divergence-free constraint is regarded as the turbulent flow field.

3. The method according to claim 1, characterized in that It also includes optimization of the latent representation, specifically: Input the turbulence field into the pre-trained VGG network, extract the mid-level features and calculate the original Gram matrix; Reconstruct the turbulence field according to the quantized potential representation, calculate its Gram matrix, and obtain the reconstructed Gram matrix; The quantized latent representation is optimized by minimizing the difference between the original Gram matrix and the reconstructed Gram matrix to obtain the optimized latent representation.

4. The method according to claim 1, wherein It also includes an adaptive parameter adjustment mechanism based on the scene type: Extract texture complexity and motion pattern features from the input video frame, construct a feature vector, and use it to determine the scene type; Preset encoding parameter templates for different scene types, including fluid decomposition threshold, neural network quantization step size, and laminar-turbulent bitrate allocation ratio; The corresponding coding parameter template is selected according to the determined scene type, and the energy threshold of the fluid dynamics decomposition, the quantization accuracy of the neural ordinary differential equation network, and the allocation strategy of the bit rate control are dynamically adjusted.

5. The method according to claim 1, characterized in that It also includes encoding using a temporal multi-scale predictive coding structure: Organize a continuous sequence of input video frames into a hierarchical temporal structure, including key frames and bidirectionally predicted frames; Perform the complete encoding process from motion field extraction to bitstream multiplexing for key frames; For bidirectional prediction frames, only the time increment of its motion field relative to the preset reference frame is encoded; The complete coding parameters of the key frames and the time increment parameters of the bidirectional prediction frames are combined to form an output bitstream with temporal scalability.

6. The method according to claim 1, characterized in that Extract pixel-level motion fields from input video frames, including: Construct a multi-scale feature pyramid of the input video frame and its adjacent frames, and iteratively estimate the optical flow field at each scale level from coarse to fine; Adopting the bidirectional optical flow consistency constraint, the symmetric error of the forward and backward optical flows in the optical flow field is calculated, and the unreliable motion estimation in the occluded area is eliminated to obtain the optimized optical flow field. Anisotropic diffusion filtering is used to impose smoothness constraints on the optimized optical flow field to generate a continuous and smooth pixel-level motion field.

Images