Video processing method and device, model training method and device, electronic equipment and storage medium

Abstract

The invention discloses a video processing method and device, a model training method and device, electronic equipment and a storage medium, and relates to the technical field of video processing and model training. Comprising a video processing method and a model training method, and the video processing method comprises the following steps: S11, receiving original video data; s12, preprocessing the original video data, including de-noising and enhancing operations; s13, inputting the preprocessed video data into a coding device, and coding through a VPU coding card; and S14, compressing the coded video data. According to the method, the video coding process is optimized through the RDOQ, the Lookahead, the HVS and the CU Level Rate Control, so that the video quality and the compression efficiency are remarkably improved; and a deep learning framework is adopted for model training, so that important features in the video data can be identified and processed more accurately, and the intelligent level of video processing is further improved.

Description

Technical Field

[0001] The present invention relates to the technical field of video processing and model training, and in particular to a video processing, model training method, device, electronic equipment and storage medium. Background Art

[0002] With the rapid development of Internet technology and multimedia applications, video has become an important medium for information dissemination. However, raw video data usually contains a lot of redundant information, and direct storage and transmission will take up a lot of resources.

[0003] Therefore, effective video compression technology is crucial to reduce storage costs and improve transmission efficiency. Although traditional video encoding methods reduce the amount of data to a certain extent, they still have problems such as low compression ratio and image quality loss.

[0004] After searching, the application scheme of Chinese patent application number CN202310271487.6 discloses a video processing model training method, device and equipment, which relates to the field of artificial intelligence technology, specifically computer vision, augmented reality, virtual reality, deep learning and other technical fields. The specific implementation of the method includes: obtaining a mask video frame, wherein the mask video frame includes a visible video block and a mask video block; inputting the mask video frame into the encoder to learn the features of the mask video frame; inputting the features of the mask video frame into the visual decoder and the motion decoder respectively, and predicting the visual codebook and hidden motion information of the mask video frame. The video processing model training method in the above patent has the following deficiencies: there are still deficiencies in the efficiency and quality of video encoding, which need to be improved. Summary of the invention

[0005] In view of the defects in the prior art, the present invention provides a video processing and model training method, including a video processing method and a model training method, wherein the video processing method comprises the following steps:

[0006] S11: receiving original video data;

[0007] S12: preprocessing the original video data, including denoising and enhancement operations;

[0008] S13: input the pre-processed video data into the encoding device and encode it through the VPU encoding card;

[0009] S14: compressing the encoded video data, and adjusting it by using the RDOQ algorithm, the Lookahead algorithm, the HVS algorithm, and the CULevel Rate Control algorithm;

[0010] S15: Output the compressed video data to a cloud server for storage and distribution;

[0011] The model training method comprises the following steps:

[0012] S21: Collect video data as training samples;

[0013] S22: preprocessing the training samples, including denoising and enhancement operations;

[0014] S23: input the preprocessed training samples into the encoding device and encode them through the VPU encoding card;

[0015] S24: compressing the encoded training samples, and adjusting them using the RDOQ algorithm, the Lookahead algorithm, the HVS algorithm, and the CULevel Rate Control algorithm;

[0016] S25: Output the compressed training samples to the cloud server for storage and distribution;

[0017] S26: Use the deep learning framework to build a neural network model, use the compressed training samples as input, and perform model training.

[0018] Preferably: in said S12, the specific steps are as follows:

[0019] Denoising: First, denoise the original video data; use filters to reduce random noise in the video; use median filters or Gaussian filters to remove salt and pepper noise;

[0020] Enhancement: After denoising, enhance the visual elements of the video to highlight important features in the video; enhance contrast by adjusting the histogram, or use a sharpening filter to enhance edge clarity.

[0021] Preferably, the RDOQ algorithm optimizes the relationship between bit rate and distortion during quantization, as follows:

[0022] S31: Initialize parameters: set an initial quantization parameter set;

[0023] S32: Calculate RD cost: For each macroblock, calculate each possible rate-distortion cost using the formula:

[0024] Among them, represents the number of coding bits under the quantization parameter, represents the distortion between the original macroblock s and the reconstructed macroblock (such as mean square error MSE), and is the Lagrange multiplier used to weigh the bit rate and distortion;

[0025] S33: Select the best quantization parameter: select the smallest one from all as the best quantization parameter;

[0026] S34: Apply quantization and perform encoding: quantize the macroblock and complete the encoding process;

[0027] In video coding, raw macroblocks are uncompressed image data extracted directly from the source video; reconstructed macroblocks are generated during the decoding process and used as reference frames within the encoder;

[0028] The original macroblock is obtained in the following manner:

[0029] Read raw frame data from the video stream;

[0030] dividing the frame into macroblocks of predetermined size;

[0031] Each macroblock contains a luminance matrix and a corresponding chrominance matrix;

[0032] The reconstruction macroblock is generated in the following way:

[0033] On the encoding side, each macroblock is transformed, quantized, and encoded;

[0034] The coded data is then decoded and reconstructed to form reconstructed macroblocks;

[0035] The reconstructed macroblocks are concatenated to form a reconstructed frame, which is used as a reference frame for subsequent motion compensation.

[0036] Preferably, the Lookahead algorithm optimizes the encoding decision of the current frame by analyzing the future frame, as follows:

[0037] S41: Frame processing: clearly dividing the video stream into a current frame group and an appearance frame group;

[0038] S42: Predictive analysis: Perform detailed analysis of the appearance of the frame group, including motion estimation ME and scene change detection, using the formula:

[0039] Where, and represent the i-th pixel value in the current and future frames respectively, and N is the total number of pixels;

[0040] S43: Coding decision optimization: adjusting the coding parameters of the current GOP based on the ME and SCD results;

[0041] S44: Encode the current GOP: encode the current GOP using the optimized parameters;

[0042] In video processing, both current frame and future frame refer to specific frames in a sequence for encoding or processing; these frames are read directly from the video file in sequence as follows:

[0043] Read video frames:

[0044] Use the video decoding library to read each frame in the video stream sequentially;

[0045] Convert frames to a format suitable for processing;

[0046] Access pixel values:

[0047] Directly index into each pixel's location via the frame buffer;

[0048] For the current frame and future frames, if the frame size is width w and height h, then the i-th pixel position is determined by its row and column number, where .

[0049] Preferably, the HVS algorithm adjusts the encoding parameters according to the visual characteristics of the human eye to optimize the subjective visual experience, as follows:

[0050] S51: Analyze visual sensitivity: determine the sensitivity of different areas in the video to the human eye and calculate the sensitivity map;

[0051] S52: Adjusting quantization parameters: According to the visual sensitivity map, adjusting quantization parameters of different regions, specifically:

[0052] Among them, is the basic quantization parameter, k is a constant;

[0053] S53: Frequency mask adjustment: adjusting frequency mask parameters to optimize coding efficiency;

[0054] S54: Contrast and saturation adjustment: adjust contrast and saturation according to HVS characteristics;

[0055] The basic quantization parameter is set as follows:

[0056] A starting QP is preset before encoding through a sequence parameter set or a picture parameter set.

[0057] Preferably: the CU Level Rate Control algorithm is used to finely control the bit rate of each coding unit to optimize the overall video quality and compression efficiency, as follows:

[0058] S61: Initialize CU bit rate: set an initial bit rate limit for each CU;

[0059] S62: Calculate CU complexity: Analyze the content complexity of each CU, including feature values ​​through texture, motion and color distribution;

[0060] S63: Dynamically adjust the bit rate: dynamically adjust the bit rate limit according to the complexity of the CU, using the formula:

[0061] Among them, is the average complexity of all CUs;

[0062] S64: Execute encoding: encode each CU according to the adjusted bit rate.

[0063] Preferably, the content complexity of each CU is analyzed as follows:

[0064] Texture Analysis:

[0065] Apply edge detection algorithm to evaluate texture intensity within CU;

[0066] Calculate the mean or variance of edge strength as an indicator of texture complexity;

[0067] Motion Estimation:

[0068] If the CU belongs to a motion compensated frame, a motion estimation algorithm is applied to estimate the motion vector within the CU;

[0069] The size and direction distribution of motion vectors can be used to measure the complexity of motion;

[0070] Color distribution analysis:

[0071] Calculate the color histogram or color saturation inside the CU;

[0072] The standard deviation or entropy of the histogram can be used to indicate color complexity.

[0073] Video processing and model training device: used for the above-mentioned video processing and model training method, specifically including:

[0074] Video acquisition module: used to receive original video data;

[0075] Preprocessing module: used to preprocess the original video data, including denoising and enhancement operations;

[0076] Encoding module: used to input the pre-processed video data into the encoding device and encode it through the VPU encoding card;

[0077] Compression module: used to compress the encoded video data, using RDOQ algorithm, Lookahead algorithm, HVS algorithm and CU Level Rate Control algorithm for adjustment;

[0078] Output module: used to output compressed video data to the cloud server for storage and distribution.

[0079] Storage medium for video processing and model training: The storage medium stores the program code of the above-mentioned video processing and model training methods.

[0080] Electronic device for video processing and model training: The electronic device is provided with the storage medium for the above-mentioned video processing and model training, and the electronic device implements video processing and model training by executing the program code of the video processing and model training method in the storage medium.

[0081] The beneficial effects of the present invention are embodied in:

[0082] 1. The present invention optimizes the video encoding process through RDOQ, Lookahead, HVS and CU Level Rate Control, significantly improving the video quality and compression efficiency; adopts a deep learning framework for model training, which can more accurately identify and process important features in video data, further improving the intelligent level of video processing.

[0083] 2. The present invention utilizes the VPU encoding card for hardware acceleration, thereby ensuring high-speed data processing capabilities and meeting the needs of real-time video processing; storage and distribution through cloud servers facilitates the management and access of video data and supports large-scale applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the description of the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0085] Figure 1 A flowchart of the video processing method proposed by the present invention;

[0086] Figure 2 This is a flow chart of the model training method proposed in the present invention. DETAILED DESCRIPTION

[0087] The following embodiments of the technical solution of the present invention are described in detail in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and are therefore only used as examples, and cannot be used to limit the protection scope of the present invention.

[0088] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by those skilled in the art to which the present invention belongs.

[0089] Embodiment 1:

[0090] Video processing and model training method, wherein the video processing method comprises the following steps:

[0091] S11: receiving original video data;

[0092] S12: preprocessing the original video data, including denoising, enhancement and other operations;

[0093] S13: input the pre-processed video data into the encoding device and encode it through the VPU encoding card;

[0094] S14: compress the encoded video data and adjust it using multiple image quality algorithms such as RDOQ (Rate Distortion Optimization Quantization) algorithm, Lookahead algorithm, HVS (Human Visual System) algorithm and CU (Coding Unit) Level Rate Control algorithm;

[0095] S15: Output the compressed video data to a cloud server for storage and distribution.

[0096] The model training method comprises the following steps:

[0097] S21: Collect video data as training samples;

[0098] S22: preprocessing the training samples, including denoising, enhancement and other operations;

[0099] S23: input the preprocessed training samples into the encoding device and encode them through the VPU encoding card;

[0100] S24: compress the encoded training samples and adjust them using multiple image quality algorithms such as RDOQ (Rate Distortion Optimization Quantization) algorithm, Lookahead algorithm, HVS (Human Visual System) algorithm and CU (Coding Unit) Level Rate Control algorithm;

[0101] S25: Output the compressed training samples to the cloud server for storage and distribution;

[0102] S26: Use the deep learning framework to build a neural network model, use the compressed training samples as input, and perform model training.

[0103] Among them, in said S12, the specific steps are as follows:

[0104] Denoising: First, denoise the original video data. Use filters to reduce random noise in the video. For example, you can use a median filter or a Gaussian filter to remove salt and pepper noise or other types of noise. Denoising not only improves the visual effect, but also helps improve the efficiency of subsequent encoding;

[0105] Enhancement: After denoising, the visual elements of the video are enhanced, such as contrast, brightness, sharpening, etc. The important features in the video are highlighted while unnecessary details are reduced, thus laying the foundation for efficient encoding. For example, the contrast can be enhanced by adjusting the histogram, or the edge definition can be enhanced by using a sharpening filter.

[0106] Among them, in said S13, the specific steps include:

[0107] Input video data: Input the pre-processed video data into the encoding device through a high-speed data interface, such as the PCIe interface, to ensure that the data transmission speed matches the VPU processing capability.

[0108] VPU Encoding: Encodes the received video data using a VPU encoding card. Splits the video frame into smaller blocks and then applies algorithms to these blocks to reduce redundant information.

[0109] Wherein, the S14 specifically includes:

[0110] RDOQ algorithm: selects the best quantization parameters to minimize video quality loss at a given bit rate; optimizes the entire video encoding process by considering the impact of each encoding decision on the final video quality and file size;

[0111] Lookahead algorithm: It is divided into two passes of encoding; the first pass analyzes the cost and reference correlation at the macroblock level, and then adjusts the quantization parameters based on this information to improve the encoding compression efficiency and video quality of the second pass;

[0112] HVS algorithm: adjusts macroblock-level quantization parameters to improve subjective quality; the system retains more information that is more important to human vision, such as edges and areas with obvious color changes, while paying less attention to subtle changes;

[0113] CU Level Rate Control algorithm: adjusts encoding parameters at the macroblock level; more accurately controls the amount of data in different areas, thereby improving overall compression efficiency and video quality.

[0114] Wherein, the S24 is specifically:

[0115] RDOQ algorithm: applies rate-distortion optimization quantization to training samples to ensure high-quality training data under limited storage and transmission resources;

[0116] Lookahead algorithm: Improves the encoding quality of training samples through a two-pass encoding process to ensure that sufficient information can be obtained from the data during model training;

[0117] HVS algorithm: adjusts encoding parameters according to the visual characteristics of the human eye, strengthens the visual features that are important for model learning, such as edges and textures, and reduces the encoding of unimportant details;

[0118] CU Level Rate Control algorithm: Controls the bit rate at the finest level to ensure that each training sample reaches the smallest possible size while meeting quality requirements.

[0119] The RDOQ algorithm optimizes the relationship between bit rate and distortion during quantization, as follows:

[0120] S31: Initialization parameters: Set an initial quantization parameter set QP0, QP1, ..., QP n ;

[0121] S32: Calculate RD cost: For each macroblock MB i , calculate each possible QP j The rate-distortion cost under , using the formula:

[0122] D(QP j )=λ×R(QP j )+D(s,c(QP j ))

[0123] Among them, R(QP j ) indicates that the quantization parameter QP j The number of coded bits under j )) represents the distortion between the original macroblock s and the reconstructed macroblock c(QPj) (e.g., mean square error MSE), and λ is the Lagrange multiplier used to weigh the bit rate and distortion;

[0124] S33: Select the best quantization parameter: from all QP j Select D(QP j )Minimum QP * As the optimal quantization parameter;

[0125] S34: Apply quantization and encode: Using QP * The macroblocks are quantized and the encoding process is completed.

[0126] In video coding, the original macroblock is the uncompressed image data extracted directly from the source video. The reconstructed macroblock is generated during the decoding process and is used as the reference frame inside the encoder and is also the decoded video output.

[0127] The original macroblock is obtained in the following manner:

[0128] Read raw frame data from the video stream;

[0129] Split the frame into macroblocks of a predetermined size (usually 16x16 pixels);

[0130] Each macroblock contains a luminance matrix and a corresponding chrominance matrix (in YUV color space);

[0131] The reconstruction macroblock is generated in the following way:

[0132] On the encoding side, each macroblock is transformed, quantized, and encoded;

[0133] The coded data is then decoded (inverse quantized and inverse transformed), reconstructed to form a reconstructed macroblock;

[0134] The reconstructed macroblocks are concatenated to form a reconstructed frame, which is used as a reference frame for subsequent motion compensation.

[0135] The Lookahead algorithm optimizes the encoding decision of the current frame by analyzing the future frame, as follows:

[0136] S41: Frame processing: clearly divide the video stream into the current frame group (GOP) and the appearance frame group;

[0137] S42: Predictive analysis: Perform detailed analysis of the appearance of the frame group, including motion estimation ME and scene change detection, using the formula:

[0138]

[0139] Among them, F current (i) and F future (i) represents the i-th pixel value in the current and future frames, respectively, and N is the total number of pixels;

[0140] S43: Coding decision optimization: adjusting the coding parameters of the current GOP based on the ME and SCD results;

[0141] S44: Encode the current GOP: Encode the current GOP using the optimized parameters.

[0142] In video processing, both current frame and future frame refer to specific frames in the encoding or processing sequence; these frames are read directly from the video file in sequence, as follows:

[0143] Read video frames:

[0144] Use a video decoding library (such as FFmpeg) to read each frame in the video stream sequentially;

[0145] Convert the frame to a format suitable for processing (e.g. YUV or RGB);

[0146] Access pixel values:

[0147] Directly index into each pixel's location via the frame buffer;

[0148] For the current frame and future frames, if the frame size is width w and height h, the i-th pixel position is determined by its row number r and column number c, where i=r×w+c.

[0149] The HVS algorithm adjusts the encoding parameters according to the visual characteristics of the human eye to optimize the subjective visual experience, as follows:

[0150] S51: Analyze visual sensitivity: determine the sensitivity of different areas in the video to the human eye and calculate the sensitivity map M(i, j);

[0151] S52: Adjusting quantization parameters: According to the visual sensitivity map, adjusting the quantization parameters QP(i, j) of different regions, specifically:

[0152] QP(i, j) = QP base -k×M(i,j)

[0153] Among them, QP base is the basic quantization parameter, k is a constant;

[0154] S53: Frequency mask adjustment: adjusting frequency mask parameters to optimize coding efficiency;

[0155] S54: Contrast and saturation adjustment: Adjust contrast and saturation according to HVS characteristics.

[0156] Among them, the basic quantization parameter QP base The settings are as follows:

[0157] A starting QP is preset before encoding through a sequence parameter set (SPS) or a picture parameter set (PPS).

[0158] The CU Level Rate Control algorithm is used to finely control the bit rate of each coding unit (CU) to optimize the overall video quality and compression efficiency, as follows:

[0159] S61: Initialize CU bit rate: Set an initial bit rate limit R for each CU cu ;

[0160] S62: Calculate CU complexity: Analyze the content complexity C of each CU cu , e.g., through feature values ​​of texture, motion, and color distribution;

[0161] S63: Dynamically adjust the bit rate: dynamically adjust the bit rate limit according to the complexity of the CU, using the formula:

[0162]

[0163] Among them, C avg is the average complexity of all CUs;

[0164] S64: Execute encoding: encode each CU according to the adjusted bit rate.

[0165] The content complexity C of each CU is analyzed. cu , as follows:

[0166] Texture Analysis:

[0167] Apply edge detection algorithms (such as Sobel operator) to evaluate the texture intensity within the CU;

[0168] Calculate the mean or variance of edge strength as an indicator of texture complexity;

[0169] Motion Estimation:

[0170] If the CU belongs to a motion compensated frame, a motion estimation algorithm is applied to estimate the motion vector within the CU;

[0171] The size and direction distribution of motion vectors can be used to measure the complexity of motion;

[0172] Color distribution analysis:

[0173] Calculate the color histogram or color saturation inside the CU;

[0174] The standard deviation or entropy of the histogram can be used to indicate color complexity.

[0175] Embodiment 2:

[0176] Video processing and model training equipment, including:

[0177] Video acquisition module: used to receive original video data;

[0178] Preprocessing module: used to preprocess the original video data, including denoising, enhancement and other operations;

[0179] Encoding module: used to input the pre-processed video data into the encoding device and encode it through the VPU encoding card;

[0180] Compression module: used to compress the encoded video data, using RDOQ (Rate Distortion Optimization Quantization) algorithm, Lookahead algorithm, HVS (Human Visual System) algorithm and CU (Coding Unit) Level Rate Control algorithm and other image quality algorithms for adjustment;

[0181] Output module: used to output compressed video data to the cloud server for storage and distribution.

[0182] Embodiment 3:

[0183] An electronic device stores program codes of the above-mentioned video processing method and model training method; the electronic device implements the technical solution of the present invention by executing these program codes.

[0184] Embodiment 4:

[0185] A storage medium stores program codes of the above-mentioned video processing method and model training method for reading and execution by electronic devices.

[0186] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. A video processing and model training method, comprising a video processing method and a model training method, characterized in that: The video processing method comprises the following steps: S11: receiving original video data; S12: preprocessing the original video data, including denoising and enhancement operations; S13: input the pre-processed video data into the encoding device and encode it through the VPU encoding card; S14: compressing the encoded video data, and adjusting it by using the RDOQ algorithm, the Lookahead algorithm, the HVS algorithm, and the CULevel Rate Control algorithm; S15: Output the compressed video data to a cloud server for storage and distribution; The model training method comprises the following steps: S21: Collect video data as training samples; S22: preprocessing the training samples, including denoising and enhancement operations; S23: input the preprocessed training samples into the encoding device and encode them through the VPU encoding card; S24: compressing the encoded training samples, and adjusting them using the RDOQ algorithm, the Lookahead algorithm, the HVS algorithm, and the CULevel Rate Control algorithm; S25: Output the compressed training samples to the cloud server for storage and distribution; S26: Use the deep learning framework to build a neural network model, use the compressed training samples as input, and perform model training.

2. The video processing and model training method according to claim 1, characterized in that: In S12, the specific steps are as follows: Denoising: First, denoise the original video data; use filters to reduce random noise in the video; use median filters or Gaussian filters to remove salt and pepper noise; Enhancement: After denoising, the visual elements of the video are enhanced to highlight the important features in the video; Increase contrast by adjusting the histogram, or use a sharpening filter to enhance edge definition.

3. The video processing and model training method according to claim 2, characterized in that: The RDOQ algorithm optimizes the relationship between bit rate and distortion during quantization, as follows: S31: Initialization parameters: Set an initial quantization parameter set QP0, QP1, ..., QP n ; S32: Calculate RD cost: For each macroblock MB i , calculate each possible QP j The rate-distortion cost under , using the formula: D(QP j )=λ×R(QP j )+D(s,c(QP j )) Among them, R(QP j ) indicates that the quantization parameter QP j The number of coded bits under j )) represents the original macroblock s and the reconstructed macroblock c (QP j ), λ is the Lagrange multiplier used to weigh the code rate and distortion; S33: Select the best quantization parameter: from all QP j Select D(QP j )Minimum QP * As the optimal quantization parameter; S34: Apply quantization and encode: Using QP * Quantize the macroblocks and complete the encoding process; In video coding, raw macroblocks are uncompressed image data extracted directly from the source video; reconstructed macroblocks are generated during the decoding process and used as reference frames within the encoder; The original macroblock is obtained in the following manner: Read raw frame data from the video stream; dividing the frame into macroblocks of predetermined size; Each macroblock contains a luminance matrix and a corresponding chrominance matrix; The reconstruction macroblock is generated in the following way: On the encoding side, each macroblock is transformed, quantized, and encoded; The coded data is then decoded and reconstructed to form reconstructed macroblocks; The reconstructed macroblocks are concatenated to form a reconstructed frame, which is used as a reference frame for subsequent motion compensation.

4. The video processing and model training method according to claim 3, characterized in that: The Lookahead algorithm optimizes the encoding decision of the current frame by analyzing future frames, as follows: S41: Frame processing: clearly dividing the video stream into a current frame group and an appearance frame group; S42: Predictive analysis: Perform detailed analysis of the appearance of the frame group, including motion estimation ME and scene change detection, using the formula: Among them, F current (i) and F future (i) represents the i-th pixel value in the current and future frames, respectively, and N is the total number of pixels; S43: Coding decision optimization: adjusting the coding parameters of the current GOP based on the ME and SCD results; S44: Encode the current GOP: encode the current GOP using the optimized parameters; In video processing, both current frame and future frame refer to specific frames in a sequence for encoding or processing; these frames are read directly from the video file in sequence as follows: Read video frames: Use the video decoding library to read each frame in the video stream sequentially; Convert frames to a format suitable for processing; Access pixel values: Directly index into each pixel's location via the frame buffer; For the current frame and future frames, if the frame size is width w and height h, the i-th pixel position is determined by its row number r and column number c, where i=r×w+c.

5. The video processing and model training method according to claim 4, characterized in that: The HVS algorithm adjusts the encoding parameters according to the visual characteristics of the human eye to optimize the subjective visual experience, as follows: S51: Analyze visual sensitivity: determine the sensitivity of different areas in the video to the human eye and calculate the sensitivity map M(i,j); S52: Adjusting quantization parameters: According to the visual sensitivity map, adjusting the quantization parameters QP(i,j) of different regions, specifically: QP(i,j)=QP base -k×M(i,j) Among them, QP base is the basic quantization parameter, k is a constant; S53: Frequency mask adjustment: adjusting frequency mask parameters to optimize coding efficiency; S54: Contrast and saturation adjustment: adjust contrast and saturation according to HVS characteristics; Among them, the basic quantization parameter QP base The settings are as follows: A starting QP is preset before encoding through a sequence parameter set or a picture parameter set.

6. The video processing and model training method according to claim 5, characterized in that: The CU Level RateControl algorithm is used to finely control the bit rate of each coding unit to optimize the overall video quality and compression efficiency, as follows: S61: Initialize CU bit rate: Set an initial bit rate limit R for each CU cu ; S62: Calculate CU complexity: Analyze the content complexity C of each CU cu , including feature values ​​distributed by texture, motion, and color; S63: Dynamically adjust the bit rate: dynamically adjust the bit rate limit according to the complexity of the CU, using the formula: Among them, C avg is the average complexity of all CUs; S64: Execute encoding: encode each CU according to the adjusted bit rate.

7. The video processing and model training method according to claim 6, characterized in that: The content complexity C of each CU is analyzed cu , as follows: Texture Analysis: Apply edge detection algorithm to evaluate texture intensity within CU; Calculate the mean or variance of edge strength as an indicator of texture complexity; Motion Estimation: If the CU belongs to a motion compensated frame, a motion estimation algorithm is applied to estimate the motion vector within the CU; The size and direction distribution of motion vectors can be used to measure the complexity of motion; Color distribution analysis: Calculate the color histogram or color saturation inside the CU; The standard deviation or entropy of the histogram can be used to indicate color complexity.

8. A device for video processing and model training, characterized in that: The video processing and model training method described in claims 1 to 7 specifically comprises: Video acquisition module: used to receive original video data; Preprocessing module: used to preprocess the original video data, including denoising and enhancement operations; Encoding module: used to input the pre-processed video data into the encoding device and encode it through the VPU encoding card; Compression module: used to compress the encoded video data, using RDOQ algorithm, Lookahead algorithm, HVS algorithm and CU Level Rate Control algorithm for adjustment; Output module: used to output compressed video data to the cloud server for storage and distribution.

9. Storage medium for video processing and model training, characterized in that: The storage medium stores the program code of the video processing and model training method described in claims 1-7.

10. Electronic equipment for video processing and model training, characterized in that: The electronic device is provided with a storage medium for video processing and model training as described in claim 9, and the electronic device implements video processing and model training by executing the program code of the video processing and model training method in the storage medium.

Images