Video data processing method, device, equipment and storage medium

Abstract

The present application provides a method, device, equipment and storage medium for processing video data. It is applied to the field of video coding technology. The method includes: if the QP values ​​of the consecutive n frames of data encoded before the current moment are all greater than the preset QP threshold, then adjust the image signal processing ISP image parameters, and process the original video data obtained at the current moment based on the adjusted ISP image parameters to obtain YUV data. In the encoding process based on the preset target bit rate, if the QP values ​​of the consecutive n frames of data obtained by encoding are all greater than the QP threshold, and the motion state of the front-end device indicates that the front-end device is stationary, the encoding reference relationship is adjusted to the first encoding reference relationship until the encoding is completed to obtain the code stream data. The above method effectively responds to the problems of network signal weakening and insufficient bandwidth by dynamically adjusting the ISP image parameters and the encoding reference relationship, thereby improving the stability of video transmission, encoding quality and user experience, while reducing bandwidth occupancy and storage costs.

Description

Technical Field

[0001] The present application relates to the field of video coding technology, and in particular to a method, apparatus, device and storage medium for processing video data. Background Art

[0002] With the advancement of society and the rise of people's safety awareness, the demand for security surveillance continues to grow. 4G dome cameras, with their advantages of wireless transmission, high-definition image quality, and intelligent analysis, are widely used in security monitoring in fields such as security, transportation, education, and healthcare. Video data encoding is particularly important when the background area changes significantly during dome camera rotation and zoom, and when 4G and wireless Wi-Fi signals are weak or unstable.

[0003] In the prior art, the network bandwidth is usually estimated through a network protocol quality service library, and then the bit rate is adjusted to ensure the encoding effect.

[0004] However, the above method forcibly reduces the encoding bit rate according to the network conditions, resulting in a significant loss of the encoded image effect, and cannot guarantee the effect and quality of video data encoding. Summary of the Invention

[0005] The present application provides a video data processing method, apparatus, device and storage medium to solve the problem that the encoding bit rate is forcibly reduced according to network conditions, resulting in obvious loss of encoded image effect and inability to guarantee the effect and quality of video data encoding.

[0006] In a first aspect, the present application provides a method for processing video data, applied to a front-end device, the method comprising:

[0007] If the QP values ​​of the consecutive n frames of data encoded before the current moment are all greater than the preset QP threshold, the image signal processing ISP image parameters are adjusted;

[0008] Processing the original video data obtained at the current moment based on the adjusted ISP image parameters to obtain YUV data corresponding to the original video;

[0009] During encoding of the YUV data based on a preset target bit rate, if QP values ​​of n consecutive frames of data obtained by encoding are all greater than the QP threshold, and the motion state of the front-end device indicates that the front-end device is stationary, the encoding reference relationship is adjusted to a first encoding reference relationship until encoding is completed to obtain code stream data, wherein the first encoding reference relationship includes: a first frame is encoded as an intra-frame coded I frame, an I frame is encoded after each first preset interval, and after each second preset interval, a forward predicted Pm frame is encoded with reference to the last I frame encoded previously. Between the first frame and the P1 frame, and between the Pm-1 frame and the Pm frame, a P frame is encoded with reference to the previous frame currently being encoded and the last I frame encoded previously to the current encoding, where m represents the mth second preset interval and is an integer greater than 2, Pm-1 represents the m-1th P frame, and Pm represents the mth P frame.

[0010] In combination with the first aspect, in some embodiments, the method further includes:

[0011] When the motion state indicates that the front-end device is moving, the encoding reference relationship is adjusted to a second encoding reference relationship until the encoding is completed to obtain the code stream data, wherein the second encoding reference relationship includes: the first frame is encoded as an I frame, an I frame is encoded after each third preset interval, and after each fourth preset interval, it is encoded as a Pn frame with reference to the last I frame encoded previously; between the first frame and the P1 frame, multiple bidirectionally predicted B frames are encoded with reference to the first frame and the P1 frame; between the Pn-1 frame and the Pn frame, multiple B frames are encoded with reference to the Pn-1 frame and the Pn frame; n represents the nth fourth preset interval, which is an integer greater than 2; the Pn-1 represents the n-1th P frame; and the Pn represents the nth P frame.

[0012] In combination with the first aspect, in some embodiments, the method further includes:

[0013] During encoding using the first encoding reference relationship, if the QP values ​​of n consecutive frames of data obtained by encoding are all greater than the QP threshold and it is detected that the current network bandwidth is less than the preset first bandwidth threshold, the first encoding reference relationship is adjusted to a third encoding reference relationship for encoding, and the third encoding reference relationship includes: encoding only the first frame as an I frame, encoding as a Pa frame with reference to the first frame after each fifth preset interval, and encoding as a p frame with reference to the previous frame currently encoded and the first frame between the first frame and the P1 frame, and between the Pa-1 frame and the Pa frame, where a represents the ath fifth preset interval and is an integer greater than 2, the Pa-1 represents the a-1th P frame, and the Pa represents the ath P frame.

[0014] In combination with the first aspect, in some embodiments, the method further includes:

[0015] During the encoding process using the third encoding reference relationship, the encoding bit rate is adjusted based on the motion ratio of the original video data, and the adjusted bit rate is used for encoding the next frame, where the motion ratio is used to indicate the rate of change of the original video data for the macroblock pixel value.

[0016] In combination with the first aspect, in some embodiments, before adjusting the encoding bit rate based on the motion ratio of the original video data, the method further includes:

[0017] Downsampling the resolution of the original video data to generate a Bitmap table;

[0018] For each macroblock corresponding to the downsampled original video data, if the change value of the pixel value of the macroblock is greater than a preset pixel value threshold, the corresponding position of the macroblock in the Bitmap table is set to 1;

[0019] If the change value of the pixel value of the macroblock is less than the pixel value threshold, the corresponding position of the macroblock in the Bitmap table is set to 0;

[0020] The proportion of 1 in the Bitmap table is determined as the motion proportion.

[0021] In combination with the first aspect, in some embodiments, if the QP values ​​of the consecutive n frames of data encoded before the current moment are all greater than the preset QP threshold, adjusting the image signal processing ISP image parameters includes:

[0022] Determining ISP target image parameters from the ISP image parameters based on the QP threshold, the ISP target image parameters including a noise reduction parameter and a sharpening parameter;

[0023] increasing the level of the noise reduction parameter;

[0024] Lower the level of the sharpening parameter.

[0025] In combination with the first aspect, in some embodiments, before adjusting the image signal processing (ISP) image parameters, the method further includes:

[0026] When the network signal weakens and the network bandwidth cannot meet the data volume of the code stream transmission, the target bit rate is adjusted based on the current network bandwidth.

[0027] In combination with the first aspect, in some embodiments, the method further includes:

[0028] If the QP values ​​of the consecutive n frames of data encoded after adjusting the encoding reference relationship are all greater than the QP threshold, the encoding frame rate and resolution are reduced.

[0029] In combination with the first aspect, in some embodiments, the method further includes:

[0030] When the motion state indicates that the front-end device is stationary and it is detected that the current network bandwidth is greater than the first bandwidth threshold and less than a preset second bandwidth threshold, the coding reference relationship is adjusted to a fourth coding reference relationship for encoding, where the fourth coding reference relationship includes: the first frame is encoded as an I frame, an I frame is encoded after each sixth preset interval, and between two I frames, a previous frame and the last I frame that has been encoded are referenced as reference frames and encoded as a P frame, and the first bandwidth threshold is less than the second bandwidth threshold.

[0031] In combination with the first aspect, in some embodiments, the method further includes:

[0032] During encoding using the second encoding reference relationship, if the QP values ​​of the encoded consecutive n frames of data are all greater than the QP threshold and it is detected that the current network bandwidth cannot meet the data volume of the code stream transmission, the motion state of the front-end device is adjusted to still.

[0033] In a second aspect, the present application provides a video data processing device, comprising:

[0034] A first adjustment module is configured to adjust image signal processing (ISP) image parameters if the QP values ​​of n consecutive frames of data encoded before the current moment are all greater than a preset QP threshold;

[0035] A processing module, configured to process the raw video data acquired at the current moment based on the adjusted ISP image parameters to obtain YUV data corresponding to the raw video;

[0036] The second adjustment module is configured to adjust the encoding reference relationship to a first encoding reference relationship until encoding is completed to obtain bitstream data if, during encoding of the YUV data based on a preset target bit rate, QP values ​​of n consecutive frames of data obtained by encoding are all greater than the QP threshold and the motion state of the front-end device indicates that the front-end device is stationary. The first encoding reference relationship includes: encoding the first frame as an intra-frame coded I frame, encoding an I frame after each first preset interval, encoding as a forward predicted Pm frame after each second preset interval with reference to the last I frame previously encoded, and encoding as a P frame with reference to the previous frame currently encoded and the last I frame previously encoded between the first frame and the P1 frame and between the Pm-1 frame and the Pm frame, where m represents the mth second preset interval and is an integer greater than 2, Pm-1 represents the m-1th P frame, and Pm represents the mth P frame.

[0037] In a third aspect, the present application provides a front-end device, comprising: a processor, and a memory, a communication interface, and an image sensor communicatively connected to the processor;

[0038] The memory stores computer-executable instructions;

[0039] The processor executes the computer-executable instructions stored in the memory to implement the video data processing method according to any one of the first aspects.

[0040] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the video data processing method described in any of the aforementioned aspects.

[0041] In a fifth aspect, the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the method for processing video data described in any of the above aspects.

[0042] The video data processing method, apparatus, device, and storage medium provided in the present application are as follows: if the QP values ​​of the consecutive n frames of data encoded before the current moment are all greater than a preset QP threshold, the image signal processing ISP image parameters are adjusted, and the original video data obtained at the current moment is processed based on the adjusted ISP image parameters to obtain YUV data corresponding to the original video. In the process of encoding the YUV data based on a preset target bit rate, if the QP values ​​of the consecutive n frames of data obtained by encoding are all greater than the QP threshold, and the motion state of the front-end device indicates that the front-end device is stationary, the encoding reference relationship is adjusted to the first encoding reference relationship until the encoding is completed and the code stream data is obtained. The above method effectively addresses the problems of network signal weakening and insufficient bandwidth by dynamically adjusting the ISP image parameters and the encoding reference relationship, thereby improving the stability of video transmission, encoding quality, and user experience, while reducing bandwidth occupancy and storage costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0044] Figure 1 A schematic diagram of the application system structure of the video data processing method provided in an embodiment of the present application;

[0045] Figure 2 A flowchart of a first embodiment of a method for processing video data provided in an embodiment of the present application;

[0046] Figure 3 is a schematic diagram of a first coding reference relationship;

[0047] Figure 4 This is a schematic diagram of the third coding reference relationship;

[0048] Figure 5 A schematic diagram of a third coding reference relationship for real-time bit rate adjustment;

[0049] Figure 6 A flowchart of a second embodiment of a method for processing video data provided in an embodiment of the present application;

[0050] Figure 7 is a schematic diagram of the second coding reference relationship;

[0051] Figure 8 A flowchart of a third embodiment of a method for processing video data provided in an embodiment of the present application;

[0052] Figure 9 Schematic diagram of the impact of noise reduction parameters on bit rate;

[0053] Figure 10 Schematic diagram of the impact of sharpening parameters on bit rate;

[0054] Figure 11 Schematic diagram of texture complexity comparison;

[0055] Figure 12 This is a comparison diagram of noise reduction level 0 and noise reduction level 100;

[0056] Figure 13 A comparison diagram of sharpening level 0 and sharpening level 100;

[0057] Figure 14 A flowchart of a fourth embodiment of a method for processing video data provided in an embodiment of the present application;

[0058] Figure 15 Schematic diagram of the fourth coding reference relationship;

[0059] Figure 16 A schematic diagram of the fourth coding reference relationship for adjustment;

[0060] Figure 17 A schematic diagram of the structure of an embodiment of a video data processing device provided in an embodiment of the present application;

[0061] Figure 18 A schematic diagram of the structure of the front-end device provided in an embodiment of the present application.

[0062] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0063] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0064] With the advancement of society and rising public awareness of safety, the demand for security surveillance continues to grow. 4G dome cameras, with their advantages of wireless transmission, high-definition image quality, and intelligent analysis, are widely used in security surveillance applications in areas such as security, transportation, education, and healthcare. Video data encoding is particularly important when the background area changes significantly during the camera's rotation or zoom, and when 4G and wireless Wi-Fi signals are weak or unstable. Existing techniques typically estimate network bandwidth using a network protocol quality service library and adjust the bitrate to ensure encoding quality. However, forcibly reducing the encoding bitrate based on network conditions results in significant loss of image quality after encoding. Furthermore, forcibly reducing the frame rate when the bitrate cannot be kept to a minimum increases the inter-frame interval and reduces the smoothness of moving objects. While balancing image quality and encoding efficiency through image preprocessing and encoding bitrate coordination can maintain good image quality at lower bitrates, over-adjusting preprocessing can lead to blurry images before encoding, resulting in significant information loss and reduced clarity, and the control methods are limited.

[0065] In response to the above problems, the present application provides a method, device, equipment and storage medium for processing video data, which improves the stability and encoding quality of video transmission under the condition of poor network signal. Specifically, in the prior art, the network bandwidth is estimated by the network protocol quality service library, and then the bit rate is adjusted to ensure the encoding effect. However, forcibly reducing the encoding bit rate according to the network conditions will result in a significant loss of the image effect after encoding, and forcibly reducing the frame rate when the bit rate cannot be suppressed will result in an increase in the frame interval and a decrease in the smoothness of the moving target. If the image preprocessing and the encoding bit rate are linked to achieve a balance between image quality and encoding efficiency, better image quality can be maintained at a lower bit rate. However, excessive preprocessing adjustment will cause the image before encoding to become blurred, and a lot of information will be lost before encoding, resulting in a decrease in clarity and a single means of control. Considering these issues, the inventors investigated whether it is possible to adjust the bitrate in real time based on network bandwidth and monitor the quantization parameter (QP) value of each frame of encoded data in real time. When the QP value of multiple consecutive frames of data reaches the QP threshold, the ISP parameters are adjusted to reduce encoding pressure. To ensure video quality, when the QP value of multiple consecutive frames of encoded data reaches the QP threshold after adjusting the image signal processor (ISP) parameters, the encoding reference relationship is adjusted based on the motion state of the front-end device, thereby reducing encoding pressure while ensuring encoding quality. Based on this, the technical solution of the present application is proposed.

[0066] Figure 1 A schematic diagram of the application system structure of the video data processing method provided in the embodiment of the present application is shown as follows: Figure 1 As shown, the system includes a front-end device such as a weak network tennis machine or a front-end camera, where weak networks include 4G, WiFi, and satellite signals. The system includes an acquisition module, a processing module, an encoder, and an output module. The acquisition module is used to acquire video data from the image sensor, the processing module is used to perform image processing on the data acquired by the acquisition module to obtain YUV data, the encoder is used to encode the YUV data, and the output module is used to transmit the bitstream data encoded by the data encoder via the 4G network. The bitstream data can be transmitted to back-end devices such as smartphones, laptops, desktop computers, servers, and the cloud.

[0067] In the above application system, in order to ensure the encoding effect and improve the video quality, the network bandwidth can be monitored in real time, and then the encoder bit rate can be adjusted in real time based on the network bandwidth, and the frame-level QP value of the data stream can be monitored. When the QP value of multiple consecutive frames of data reaches the preset threshold, it means that the current encoding pressure is large, and the processing parameters of the processing module are adjusted, and the encoding reference relationship is adjusted, or the encoding resolution and frame rate are reduced until the network bandwidth is met.

[0068] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0069] Figure 2 The flowchart of the first embodiment of the method for processing video data provided in the embodiment of the present application is as follows: Figure 2 As shown, the method can be applied to a front-end device, and the method specifically includes:

[0070] S201: If the QP values ​​of the consecutive n frames of data encoded before the current moment are all greater than a preset QP threshold, adjust the image signal processing (ISP) image parameters.

[0071] In this step, in order to monitor the encoding pressure during the encoding process in real time, the QP values ​​of the consecutive n frames of data encoded before the current moment can be obtained in real time.

[0072] Specifically, the QP value is a key parameter that affects video quality and file size, and thus can be used to judge encoding pressure.

[0073] For example, for most encoders, detailed logging is enabled. This can usually be achieved by setting specific parameters in the encoding command. For example, in FFmpeg, you can use -loglevel debug or -loglevel verbose to enable more detailed log output. These logs will contain QP value information for each frame. After obtaining the log or output file, you can analyze the log or output file, either manually or by writing a script to automatically extract the relevant QP value. The above is only an example of obtaining the QP value, and the embodiment of the present application does not specifically limit the specific implementation method for obtaining the QP value.

[0074] It should be noted that the obtained QP value is used to determine the encoding pressure. The above n is an integer greater than 1. However, if the QP value obtained is for one frame, it is not convincing. Therefore, in actual application scenarios, n can be set to an integer greater than 3. It can be set according to the actual application scenario and is not specifically limited in the embodiments of this application.

[0075] After obtaining the QP value of n consecutive frames of data, the QP value of each frame is compared with the preset QP threshold. If the QP values ​​of n consecutive frames of data are all greater than the preset QP threshold, it means that the current encoding pressure is too high. In this case, the ISP image parameters can be adjusted to reduce the image clarity, thereby reducing the texture complexity of the data used for encoding, thereby reducing the encoding pressure.

[0076] Specifically, based on the QP threshold, the ISP target image parameters are determined in the ISP image parameters, and the ISP target image parameters include noise reduction parameters and sharpening parameters, thereby increasing the level of the noise reduction parameters and reducing the level of the sharpening parameters.

[0077] S202: Processing the original video data obtained at the current moment based on the adjusted ISP image parameters to obtain YUV data corresponding to the original video.

[0078] In this step, after adjusting the ISP image parameters, the original video data is processed based on the adjusted ISP image parameters to obtain YUV data, wherein the YUV data is used for encoding.

[0079] Specifically, the ISP is used to convert the raw video data captured by the sensor (usually in RAW format) into data more suitable for subsequent processing and display (such as YUV format). YUV is a color encoding method where Y represents luminance (Luminance or Luma) and U and V represent chrominance (Chrominance or Chroma) components, commonly used for video compression and transmission. The specific processing steps of the ISP include:

[0080] Denoising: Reducing random noise in an image.

[0081] Defect Correction: Repairs dead pixels or defects in image sensors.

[0082] White Balance: Adjusts the image colors to reflect the true colors of the light sources in the scene.

[0083] Auto Exposure: Adjusts the brightness of the image to ensure it is not overexposed or underexposed.

[0084] Color Correction: Adjust the color saturation, contrast, and hue of an image.

[0085] Sharpening: Enhances the edges and details of an image to make it appear clearer.

[0086] Demosaicing: Converting RAW data (usually a Bayer pattern) into a full-color image.

[0087] After processing by the ISP, the data is usually in RGB format (although different color spaces may be used for processing within the ISP). The RGB data then needs to be converted to YUV format for further video processing, compression, or display. Specifically, the conversion from RGB to YUV is usually performed using standard mathematical formulas that convert the RGB components into Y (luminance) and U, V (chrominance) components. These conversions can be linear or nonlinear, depending on the required color accuracy and computational efficiency. This is not specifically limited in the embodiments of the present application.

[0088] It should be noted that there are many variants of the YUV format (such as YUV420, YUV422, YUV444, etc.). These variants differ in the sampling and storage methods of the chrominance components. It is necessary to select the appropriate format according to the specific application scenario.

[0089] S203: During the YUV data encoding process based on a preset target bit rate, if the QP values ​​of n consecutive frames of encoded data are all greater than the QP threshold, and the motion state of the front-end device indicates that the front-end device is stationary, the encoding reference relationship is adjusted to the first encoding reference relationship until the encoding is completed to obtain the bitstream data.

[0090] In this step, after adjusting the ISP parameters, encoding continues. If the QP value of n consecutive frames of data is still greater than the QP threshold during encoding, the encoding reference relationship can be adjusted based on the motion state of the front-end device. Finally, encoding is completed and the bitstream data is output. The motion state of the front-end device is used to indicate that the front-end device is stationary.

[0091] Specifically, when the motion state indicates that the front-end device is stationary, such as a weak tennis ball machine, most areas of the ball machine remain unchanged when it is stationary (not rotating). In order to add more reference information, it can be determined to use a first coding reference relationship. The first coding reference relationship includes: the first frame is encoded as an I frame, an I frame is encoded after each first preset interval, and after each second preset interval, it is encoded as a Pm frame with reference to the last I frame encoded previously. Between the first frame and the P1 frame, and between the Pm-1 frame and the Pm frame, a p frame is encoded with reference to the previous frame currently encoded and the last I frame encoded previously to the current encoding. m represents the mth second preset interval and is an integer greater than 2. Pm-1 represents the m-1th P frame, and Pm represents the mth P frame.

[0092] For example, Figure 3 is a schematic diagram of the first coding reference relationship, such as Figure 3 As shown, the P frame encoded in the first coding reference relationship not only refers to the previous frame but also refers to the I or P frame. A P frame and an I frame are added at a certain interval to reduce the residual of the subsequent P frame. Among them, the encoding method of the P frame and the P frame is the same, the difference is that the reference frame is different. This encoding method can optimize the transmission efficiency while maintaining the video quality. It should be noted that, with reference to Figure 3 When encoding according to the above reference relationship, it is also necessary to preset the interval encoding of I-frames. Taking the common frame rate of 25fps as an example, the I-frame interval is 50 (equal to one I-frame every 2 seconds), and there are 49 frames between two I-frames. Each I-frame only serves as an intra-frame reference and does not reference other frames. In this frame structure, the I-frame interval is 200, and the P-frame interval is 50. P-frames only reference I-frames, while other frames are ordinary P-frames that reference the previous frame and I-frame. With this reference structure, the user can click on any frame and simply find the P-frame that precedes it to decode it sequentially. A maximum of 49 frames can be decoded to display the current frame.

[0093] In one possible implementation, if the QP values ​​of n consecutive frames of data encoded after adjusting the encoding reference relationship are still greater than the QP threshold, the encoding frame rate and resolution can be reduced until the network bandwidth is met.

[0094] Optionally, when the network signal weakens and the network bandwidth cannot meet the data volume of the stream transmission, the target bit rate is adjusted based on the current network bandwidth.

[0095] In this step, the network bandwidth is monitored in real time. When it is detected that the signal is weakening and the network bandwidth of the communication link cannot meet the data volume required for the code stream transmission, the bit rate value configured for the encoder is reduced. After the bit rate value is reduced, the loss after encoding will theoretically increase and the image effect will deteriorate. Therefore, while reducing the code rate, the frame-level QP value of the encoded data must be monitored in real time to ensure video quality.

[0096] Specifically, network signal monitoring involves analyzing and measuring the raw signal received from the network interface to obtain signal strength metrics such as RSSI (Signal Strength Indicator) and Signal-to-Noise Ratio (SNR). The system can preset a series of signal strength thresholds based on network standards, device performance, and application scenario requirements. The real-time measured signal strength is compared with the preset thresholds. If it falls below a certain threshold, the signal is considered weak.

[0097] Network bandwidth can be assessed by measuring the amount of data successfully transmitted per unit time. This typically involves timing and counting transmitted packets. Common measurement methods include throughput testing, which involves sending a large amount of data within a specified timeframe and measuring the amount of data received.

[0098] You can also determine the required bitstream data volume based on application requirements. You also need to consider data redundancy, coding efficiency, and network protocol overhead to calculate the actual network bandwidth required.

[0099] The actual network bandwidth is then compared with the required data volume for the bitstream transmission. If the actual bandwidth is lower than the required bandwidth, it is considered that the network bandwidth of the communication link cannot meet the data volume required for the bitstream transmission.

[0100] When a decrease in network bandwidth is detected, the encoding bit rate is lowered to reduce data transmission volume and avoid network congestion and buffering delays.

[0101] On the contrary, when the network bandwidth increases, the encoding bit rate can be appropriately increased to improve video quality.

[0102] It should be noted that the encoding bit rate has a pre-set bit rate upper limit value, and during the dynamic adjustment process, it cannot exceed the bit rate upper limit value.

[0103] Optionally, during the encoding process using the first encoding reference relationship, the QP value of the n consecutive frames of data obtained after encoding is detected in real time. If it is still greater than the QP threshold, and it is detected that the current network bandwidth is less than the preset first bandwidth threshold, the first encoding reference relationship is adjusted to a third encoding reference relationship.

[0104] Specifically, the third coding reference relationship includes: encoding as an I frame only in the first frame, encoding as a Pa frame with reference to the first frame after each fifth preset interval, and encoding as a p frame with reference to the previous frame currently encoded and the first frame between the first frame and the P1 frame, a represents the ath fifth preset interval, which is an integer greater than 2, Pa-1 represents the a-1th P frame, and Pa represents the ath P frame.

[0105] For example, Figure 4is a schematic diagram of the third coding reference relationship, such as Figure 4 As shown above Figure 3 The frame structure can solve the problem of reducing residual errors and quickly viewing recordings, but an I frame will still be generated every 200 frames. In extremely low-bandwidth scenarios, it is still easy to cause instantaneous bandwidth congestion, resulting in image freezes. When the QP and network bandwidth thresholds are met, I frames can no longer be generated regularly.

[0106] It should be noted that an I-frame is generated immediately in the following situations: the original frame structure cannot be decoded normally due to reasons such as packet loss, in which case an I-frame is generated immediately; the scene detection results for N consecutive seconds indicate that the current scene should change, and the original I-frame is no longer a reference, in which case an I-frame is also generated immediately.

[0107] In one possible design, during encoding with a third coding reference relationship, the frame structure can also dynamically adjust the bit rate value at the current moment according to the motion ratio of the current scene, where the motion ratio is used to indicate the rate of change of the original video data for the macroblock pixel value.

[0108] For example, Figure 5 A schematic diagram of the third coding reference relationship for real-time bit rate adjustment, such as Figure 5 As shown in the figure, when the motion ratio exceeds the set threshold, a high bitrate value is set immediately, and the next frame is required to take effect immediately, without waiting for the next I frame or large P frame to take effect. The final size structure diagram of each frame is as follows Figure 5 As shown, the frames in the dotted box correspond to frames where the motion ratio exceeds the threshold and a high bit rate is set. During this period, the bit rate is high and the encoding effect is good. The frames outside the dotted box correspond to frames where the motion ratio is low.

[0109] The video data processing method provided in this embodiment is as follows: if the QP values ​​of the consecutive n frames of data encoded before the current moment are all greater than a preset QP threshold, the image signal processing ISP image parameters are adjusted, and the original video data obtained at the current moment is processed based on the adjusted ISP image parameters to obtain YUV data corresponding to the original video. In the process of encoding the YUV data based on a preset target bit rate, if the QP values ​​of the consecutive n frames of data obtained by encoding are all greater than the QP threshold, and the motion state of the front-end device indicates that the front-end device is stationary, the encoding reference relationship is adjusted to the first encoding reference relationship until the encoding is completed and the code stream data is obtained. The above method effectively addresses the problems of network signal weakening and insufficient bandwidth by dynamically adjusting the ISP image parameters and the encoding reference relationship, thereby improving the stability of video transmission, encoding quality, and user experience, while reducing bandwidth occupancy and storage costs.

[0110] Figure 6The flowchart of the second embodiment of the method for processing video data provided in the embodiment of the present application is as follows: Figure 6 As shown, the method includes:

[0111] S601: If the QP values ​​of the n consecutive frames of data encoded before the current moment are all greater than a preset QP threshold, adjust the image signal processing (ISP) image parameters.

[0112] S602: Processing the original video data obtained at the current moment based on the adjusted ISP image parameters to obtain YUV data corresponding to the original video.

[0113] Step S601 and step 602 are implemented in the same manner as step S201 and step S202 in the aforementioned embodiment, and are not described again here.

[0114] S603: During the YUV data encoding process based on a preset target bit rate, if the QP values ​​of n consecutive frames of encoded data are all greater than the QP threshold, and the motion state of the front-end device indicates that the front-end device is moving, the encoding reference relationship is adjusted to a second encoding reference relationship until the encoding is completed to obtain the bitstream data.

[0115] In this step, when the motion state indicates that the front-end device is moving, such as in a weak tennis ball machine, most of the background area changes when the ball machine is moving (rotating). The conventional reference relationship has large residual errors, resulting in high encoding pressure and poor image quality. In addition, the encoding bitstream data volume may be large during movement, causing network packet loss and other problems. The second encoding reference relationship is then used for encoding.

[0116] Among them, the second coding reference relationship includes: the first frame is encoded as an I frame, an I frame is encoded after each third preset interval, and after each fourth preset interval, it is encoded as a Pn frame with reference to the last I frame encoded previously. Between the first frame and the P1 frame, it is encoded as multiple bidirectionally predicted B frames with reference to the first frame and the P1 frame. Between the Pn-1 frame and the Pn frame, it is encoded as multiple B frames with reference to the Pn-1 frame and the Pn frame. n represents the nth fourth preset interval, which is an integer greater than 2, Pn-1 represents the n-1th P frame, and Pn represents the nth P frame.

[0117] For example, Figure 7 is a schematic diagram of the second coding reference relationship, such as Figure 7 As shown, B frames reference previous and subsequent frames, making it easier to find matching blocks during motion and reducing residual errors. This makes B frames particularly small, reduces encoding pressure, and improves overall video encoding performance. At the same time, all P frames reference only I frames. This way, even if a P frame is lost, subsequent P frames can still be used. Because 4G networks may have poor signal quality, this reference relationship can help prevent packet loss.

[0118] The background changes dramatically during the rotation of the dome camera, increasing the amount of data that needs to be encoded. When the network bandwidth is low, slow transmission and severe packet loss can cause image freezes and distorted screens, resulting in a very poor user experience. Furthermore, the slip rings that rotate the dome camera have a lifespan issue (related to the number of rotations). When the network bandwidth is low, rotating the dome camera not only wastes the lifespan of the slip rings but also poors the preview experience. If the QP and network bandwidth thresholds are met, the dome camera can be prevented from rotating.

[0119] It should be noted that the preset interval in this embodiment is determined according to the encoding frame rate and the actual application scenario.

[0120] The video data processing method provided in this embodiment adjusts the image signal processing (ISP) image parameters if the QP values ​​of n consecutive frames of data encoded before the current moment are all greater than a preset QP threshold, and processes the original video data acquired at the current moment based on the adjusted ISP image parameters to obtain YUV data corresponding to the original video. During the encoding of the YUV data based on a preset target bit rate, if the QP values ​​of the encoded consecutive n frames of data are all greater than the QP threshold, and the motion state of the front-end device indicates that the front-end device is in motion, the encoding reference relationship is adjusted to a second encoding reference relationship until the encoding is completed to obtain the bitstream data. The above method optimizes the video encoding process, thereby adapting to different motion scenarios, increasing encoding flexibility, and achieving the effects of improving encoding efficiency, reducing data volume, and maintaining video quality.

[0121] Figure 8 The flowchart of the third embodiment of the method for processing video data provided in the embodiment of the present application is as follows: Figure 8 As shown, based on the above embodiment, step S201 specifically includes:

[0122] S801: Based on the QP threshold, determine the ISP target image parameters from the ISP image parameters.

[0123] In this step, to ensure coding pressure, the impact of each ISP image parameter on the average bitrate of the output stream can be analyzed based on a fixed QP, thereby determining the initial ISP target image parameters. Through this determination, the ISP target image parameters include noise reduction parameters and sharpening parameters.

[0124] For example, Figure 9 This is a schematic diagram of the impact of noise reduction parameters on bit rate. Figure 10 This is a diagram showing the effect of sharpening parameters on bit rate. Figure 9 As shown in the figure, by evaluating the noise reduction parameters in a fixed QP manner, it can be concluded that the greater the noise reduction level, the lower the bit rate, as shown in the figure. Figure 10As shown in FIG, by evaluating the sharpening parameters using a fixed QP method, it can be concluded that the greater the sharpening level, the greater the bit rate.

[0125] S802: Raise the level of the noise reduction parameter.

[0126] S803: Reduce the level of sharpening parameters.

[0127] In order to reduce the texture complexity of the YUV data and reduce the encoding pressure, the level of the noise reduction parameter can be increased and / or the level of the sharpening parameter can be reduced.

[0128] Exemplarily, the texture complexity of YUV data can be measured by standard deviation. Exemplarily, in units of macroblocks, the texture complexity of a macroblock corresponds to the standard deviation of the brightness component of a 16*16 pixel area. The larger the value, the more complex the texture, and the smaller the value, the simpler the texture.

[0129] Figure 11 A schematic diagram of texture complexity comparison, such as Figure 11 As shown, for a white wall (box 1), the values ​​of each pixel in a macroblock are very close, and the difference between the value corresponding to each pixel and the mean is very small, so the calculated standard deviation will also be very small, so the texture of the white wall is simple.

[0130] Leaves / grass (box 2), the values ​​of each pixel in a macroblock vary greatly, some are bright, some are dark, some are white, and some are green. The difference between the value corresponding to each pixel and the mean is very large, so the calculated standard deviation will also be very large, so the texture of leaves / grass is complex.

[0131] Figure 12 This is a comparison diagram of noise reduction level 0 and noise reduction level 100. Figure 13 This is a comparison diagram of sharpening level 0 and sharpening level 100, combined with Figure 12 and Figure 13 As shown in the figure, by adjusting the noise reduction intensity or sharpening intensity, the image will appear blurred. Correspondingly, the difference between each pixel becomes smaller after being processed by the noise and sharpening algorithms, and the corresponding texture complexity will also decrease.

[0132] Optionally, when images contain significant random noise (such as luminance noise and dark noise), increasing the noise reduction level can effectively reduce this noise, resulting in a cleaner image. In applications requiring high-quality video output, such as HD movies and professional surveillance, where image purity is paramount, appropriately increasing the noise reduction level can improve overall video quality. When the encoder's performance is limited, increasing the noise reduction level to reduce image complexity can reduce the encoder's burden and improve encoding efficiency.

[0133] If an image has been oversharpened, resulting in jagged edges and artifacts, the sharpening level should be reduced. In applications where detail preservation is less critical, such as live streaming and video conferencing, lowering the sharpening level can reduce high-frequency information in the image, thereby reducing encoding pressure. In some cases, excessively high sharpening parameters may introduce additional encoding noise. Lowering the sharpening level can reduce this noise and improve encoded image quality.

[0134] When image quality is already low and there is noticeable noise, it's necessary to simultaneously increase the noise reduction level to reduce the noise and appropriately reduce the sharpening level to avoid introducing additional noise or aliasing. When encoder performance is limited but image quality must be maintained, it's necessary to adjust both noise reduction and sharpening parameters to find the optimal balance. This means reducing image noise while avoiding quality issues caused by over-sharpening. In certain application scenarios (such as night mode for security surveillance), adjusting both noise reduction and sharpening parameters may be necessary to ensure both image clarity and noise levels are within acceptable ranges.

[0135] The embodiments of the present application do not specifically limit the implementation of specific adjustment parameters for reducing texture complexity.

[0136] The specific implementation method of reducing the noise reduction level and increasing the sharpness level is not specifically limited in this embodiment of the application.

[0137] The video data processing method provided in this embodiment determines ISP target image parameters from the ISP image parameters based on a QP threshold, increases the noise reduction parameter level, and decreases the sharpening parameter level. This QP threshold-based ISP image parameter adjustment method improves image quality, adapts to different network environments, enhances user experience, and conserves bandwidth resources.

[0138] Figure 14 This is a flow chart of a fourth embodiment of the method for processing video data provided in the embodiment of the present application, as shown in FIG. Figure 14 As shown, based on the above embodiment, the method further includes:

[0139] S1401: Downsample the resolution of the original video data to generate a Bitmap table.

[0140] S1402: For each macroblock corresponding to the downsampled original video data, if the change value of the pixel value of the macroblock is greater than a preset pixel value threshold, the corresponding position of the macroblock in the Bitmap table is set to 1.

[0141] S1403: If the change value of the pixel value of the macroblock is less than the pixel value threshold, the corresponding position of the macroblock in the Bitmap table is set to 0.

[0142] S1404: Determine the proportion of 1 in the Bitmap table as the motion proportion.

[0143] For example, the sensor's captured resolution is downsampled by 1 / 16 in width and height, and a bitmap table of (original image width / 16)*(original image height / 16) is generated, with each element in the table occupying only 1 bit. A frame difference is then applied to the downsampled image to determine the pixel value change for the 16*16 macroblock. If the change reaches a set threshold, the macroblock is considered to be in motion, and the corresponding position in the bitmap table is set to 1; otherwise, it is set to 0. Finally, a bitmap table is generated that represents both motion and stillness for all 16*16 macroblocks. The sum of the number of elements with a value of 1 is used to calculate the motion percentage.

[0144] The video data processing method provided in this embodiment downsamples the resolution of the original video data and generates a bitmap table. For each macroblock corresponding to the downsampled original video data, if the change in the pixel value of the macroblock is greater than a preset pixel value threshold, the corresponding position of the macroblock in the bitmap table is set to 1. If the change in the pixel value of the macroblock is less than the pixel value threshold, the corresponding position of the macroblock in the bitmap table is set to 0. The proportion of 1s in the bitmap table is determined as the motion proportion. By adjusting the encoding bitrate of the next frame in real time based on the motion proportion, the encoding effect is improved and video quality is guaranteed.

[0145] Optionally, based on the foregoing embodiment, when the motion state indicates that the front-end device is stationary and it is detected that the current network bandwidth is greater than the first bandwidth threshold and less than the preset second bandwidth threshold, the coding reference relationship can be adjusted to a fourth coding reference relationship.

[0146] Specifically, the fourth coding reference relationship includes: the first frame is encoded as an I frame, an I frame is encoded after each sixth preset interval, and between two I frames, the previous frame and the last I frame that has been encoded are referenced as reference frames and encoded as a P frame, and the first bandwidth threshold is less than the second bandwidth threshold.

[0147] For example, Figure 15 is a schematic diagram of the fourth coding reference relationship, such as Figure 15 As shown in FIG, the original coding reference relationship is blocked by a moving target in the originally static area, and the matching block cannot be found by only referring to part of the previous frame, resulting in a large coding residual of the current frame. If the previous frame is referred to at the same time as the earlier I frame, the matching block can be found more easily, thereby reducing the residual.

[0148] Optional, Figure 16 This is a schematic diagram of the fourth coding reference relationship adjusted in real time, such as Figure 16 As shown above Figure 15The medium reference relationship can reduce the coding residual, but the I-frame interval is still 50. Every 50 frames, a large I-frame will still be generated, causing regular bit rate fluctuations. This is not friendly to products with weak networks or traffic-saving products. In order to reduce bit rate fluctuations, the I-frame interval can be enlarged, and the original I-frame interval of 50 can be changed to an I-frame interval of 200. However, this reference method will cause the display time to be too long when viewing the video. If the user clicks on the N+199 frame when viewing the video, it will be necessary to start from the N frame and decode 199 frames to preview the frame image clicked by the user. In this way, the user needs to wait too long, and the user experience is not good. In this way, the frame reference relationship can be adjusted to Figure 3 The reference relationship shown can solve the problem of users quickly decoding video.

[0149] Figure 17 A schematic diagram of the structure of an embodiment of a video data processing device provided in an embodiment of the present application is shown as follows: Figure 17 As shown, the video data processing device 1700 includes:

[0150] The first adjustment module 1701 is configured to adjust image signal processing (ISP) image parameters if the QP values ​​of n consecutive frames of data encoded before the current moment are all greater than a preset QP threshold.

[0151] The processing module 1702 is used to process the original video data obtained at the current moment based on the adjusted ISP image parameters to obtain YUV data corresponding to the original video.

[0152] The second adjustment module 1703 is configured to adjust the encoding reference relationship to a first encoding reference relationship during encoding of YUV data based on a preset target bit rate, if QP values ​​of n consecutive frames of data obtained by encoding are all greater than a QP threshold and the motion state of the front-end device indicates that the front-end device is stationary, until encoding is completed to obtain bitstream data, wherein the first encoding reference relationship includes: encoding the first frame as an intra-frame coded I frame, encoding an I frame after each first preset interval, encoding as a forward predicted Pm frame after each second preset interval with reference to the last I frame previously encoded, and encoding as a P frame with reference to the previous frame currently encoded and the last I frame previously encoded between the first frame and the P1 frame, and between the Pm-1 frame and the Pm frame, where m represents the mth second preset interval and is an integer greater than 2, Pm-1 represents the m-1th P frame, and Pm represents the mth P frame.

[0153] Optionally, the second adjustment module 1703 may also be used to:

[0154] When the motion state indicates that the front-end device is moving, the encoding reference relationship is adjusted to the second encoding reference relationship until the encoding is completed to obtain the code stream data. The second encoding reference relationship includes: the first frame is encoded as an I frame, an I frame is encoded after each third preset interval, and after each fourth preset interval, it is encoded as a Pn frame with reference to the last I frame encoded previously. Between the first frame and the P1 frame, multiple B frames are encoded with reference to the first frame and the P1 frame. Between the Pn-1 frame and the Pn frame, multiple B frames are encoded with reference to the Pn-1 frame and the Pn frame. n represents the nth fourth preset interval and is an integer greater than 2. Pn-1 represents the n-1th P frame and Pn represents the nth P frame.

[0155] Optionally, the second adjustment module 1703 may also be used to:

[0156] During the encoding process using the first coding reference relationship, if the QP values ​​of n consecutive frames of data obtained by encoding are all greater than the QP threshold and it is detected that the current network bandwidth is less than the preset first bandwidth threshold, the first coding reference relationship is adjusted to a third coding reference relationship for encoding, and the third coding reference relationship includes: only encoding the first frame as an I frame, encoding as a Pa frame with reference to the first frame after each fifth preset interval, and encoding as a p frame with reference to the previous frame currently encoded and the first frame between the first frame and the P1 frame, and the Pa-1 frame and the Pa frame, where a represents the ath fifth preset interval and is an integer greater than 2, Pa-1 represents the a-1th P frame, and Pa represents the ath P frame.

[0157] Optionally, the second adjustment module 1703 may also be used to:

[0158] During the encoding process using the third encoding reference relationship, the encoding bit rate is adjusted based on the motion ratio of the original video data, and the adjusted bit rate is used for encoding the next frame. The motion ratio is used to indicate the rate of change of the original video data for the macroblock pixel value.

[0159] Optionally, the processing module 1702 may also be used to:

[0160] Downsample the resolution of the original video data to generate a Bitmap table;

[0161] For each macroblock corresponding to the downsampled original video data, if the change in the pixel value of the macroblock is greater than the preset pixel value threshold, the corresponding position of the macroblock in the Bitmap table is set to 1;

[0162] If the change in the pixel value of the macroblock is less than the pixel value threshold, the corresponding position of the macroblock in the Bitmap table is set to 0;

[0163] The proportion of 1 in the Bitmap table is determined as the motion proportion.

[0164] Optionally, the first adjustment module 1701 is specifically configured to:

[0165] Based on the QP threshold, ISP target image parameters are determined from the ISP image parameters, where the ISP target image parameters include noise reduction parameters and sharpening parameters;

[0166] Increase the level of noise reduction parameters;

[0167] Reduce the level of sharpening parameters.

[0168] Optionally, the first adjustment module 1701 is further configured to:

[0169] When the network signal weakens and the network bandwidth cannot meet the data volume of the stream transmission, the target bit rate is adjusted based on the current network bandwidth.

[0170] If the QP values ​​of the encoded n consecutive frames of data after adjusting the encoding reference relationship are all greater than the QP threshold, the encoding frame rate and resolution are reduced.

[0171] Optionally, the second adjustment module 1703 may also be used to:

[0172] When the motion state indicates that the front-end device is stationary and it is detected that the current network bandwidth is greater than the first bandwidth threshold and less than the preset second bandwidth threshold, the coding reference relationship is adjusted to a fourth coding reference relationship for encoding. The fourth coding reference relationship includes: the first frame is encoded as an I frame, an I frame is encoded after each sixth preset interval, and between two I frames, the previous frame and the last I frame that has been encoded are referenced as reference frames and encoded as a P frame. The first bandwidth threshold is less than the second bandwidth threshold.

[0173] Optionally, the second adjustment module 1703 may also be used to:

[0174] During encoding using the second encoding reference relationship, if the QP values ​​of the encoded n consecutive frames of data are all greater than the QP threshold and it is detected that the current network bandwidth cannot meet the data volume of the code stream transmission, the motion state of the front-end device is adjusted to still.

[0175] Figure 18 A schematic diagram of the structure of the front-end device provided in the embodiment of the present application is shown as follows: Figure 18 As shown, the front-end device 1800 includes: a processor 1802, a memory 1801 communicatively connected to the processor 1802, a communication interface 1803, and an image sensor 1804;

[0176] Memory 1801 stores computer-executable instructions;

[0177] The processor 1802 executes the computer-executable instructions stored in the memory 1801 to implement the video data processing method in any method embodiment.

[0178] The communication interface 1803 is used for communication with external devices.

[0179] The image sensor 1804 is used to collect video data in real time.

[0180] An embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to execute the video data processing methods provided in the various embodiments described above.

[0181] The computer-readable storage medium mentioned above may be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic storage, flash memory, magnetic disk, or optical disk. The computer-readable storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0182] Optionally, a readable storage medium is coupled to the processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium may also be an integral part of the processor. The processor and the readable storage medium may be located in an application-specific integrated circuit (ASIC). Of course, the processor and the readable storage medium may also exist as discrete components in the device.

[0183] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when at least one processor executes the computer program, it can implement the technical solution provided by any of the above method embodiments.

[0184] In this application, "at least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship; in a formula, the character " / " indicates that the previous and next associated objects are in a "division" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or plural.

[0185] It is understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of description and are not intended to limit the scope of the embodiments of the present application. In the embodiments of the present application, the order of the sequence numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0186] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0187] It should be understood that the present application is not limited to the exact structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for processing video data, characterized in that: Applied to a front-end device, the method comprises: If the QP values ​​of the consecutive n frames of data encoded before the current moment are all greater than the preset QP threshold, the image signal processing ISP image parameters are adjusted; Processing the original video data acquired at the current moment based on the adjusted ISP image parameters to obtain YUV data corresponding to the original video; In the process of encoding the YUV data based on the preset target bit rate, if the QP values ​​of the encoded continuous n frames of data are all greater than the QP threshold, and the motion state of the front-end device indicates that the front-end device is stationary, the encoding reference relationship is adjusted to the first encoding reference relationship until the encoding is completed to obtain the code stream data, and the first encoding reference relationship includes: the first frame is encoded as an intra-frame coded I frame, an I frame is encoded after each first preset interval, and after each second preset interval, the last I frame encoded in the front is encoded as a forward prediction P frame. m Frame, between the first frame and P1 frame, P m-1 Frame and P m Between frames, the previous frame of the current encoding and the last I frame encoded before the current encoding are encoded as p frames, m represents the mth second preset interval, which is an integer greater than 2, and the P m-1 represents the m-1th P frame, the P m Indicates the mth P frame.

2. The method according to claim 1, characterized in that The method further comprises: When the motion state indicates that the front-end device is moving, the coding reference relationship is adjusted to a second coding reference relationship until the coding is completed to obtain the code stream data, wherein the second coding reference relationship includes: the first frame is coded as an I frame, an I frame is coded after each third preset interval, and a P frame is coded after each fourth preset interval with reference to the last I frame coded previously. n frame, between the first frame and the P1 frame, the first frame and the P1 frame are referenced to encode into a plurality of bidirectional prediction B frames, between the P n-1 Frame and P n Between frames, refer to the P n-1 Frame and the P n The frame is encoded as a plurality of B frames, n represents the nth fourth preset interval, which is an integer greater than 2, and the P n-1 represents the n-1th P frame, the P n Indicates the nth P frame.

3. The method according to claim 1, characterized in that The method further comprises: In the encoding process using the first encoding reference relationship, if the QP values ​​of the encoded consecutive n frames of data are all greater than the QP threshold and it is detected that the current network bandwidth is less than the preset first bandwidth threshold, the first encoding reference relationship is adjusted to a third encoding reference relationship for encoding, and the third encoding reference relationship includes: only encoding the first frame as an I frame, and encoding the first frame as a P frame after each fifth preset interval with reference to the first frame. a Frame, between the first frame and P1 frame, P a-1 Frame and P a Between frames, the previous frame and the first frame of the current encoding are encoded as p frames, a represents the ath fifth preset interval, which is an integer greater than 2, and the P a-1 represents the a-1th P frame, the P a Indicates the ath P frame.

4. The method according to claim 3, characterized in that The method further comprises: During the encoding process using the third encoding reference relationship, the encoding bit rate is adjusted based on the motion ratio of the original video data, and the adjusted bit rate is used for encoding the next frame, wherein the motion ratio is used to indicate the rate of change of the original video data for the macroblock pixel value.

5. The method according to claim 4, characterized in that Before adjusting the encoding bit rate based on the motion ratio of the original video data, the method further includes: Downsampling the resolution of the original video data to generate a Bitmap table; For each macroblock corresponding to the downsampled original video data, if the change value of the pixel value of the macroblock is greater than a preset pixel value threshold, the corresponding position of the macroblock in the Bitmap table is set to 1; If the change value of the pixel value of the macroblock is less than the pixel value threshold, the corresponding position of the macroblock in the Bitmap table is set to 0; The proportion of 1 in the Bitmap table is determined as the motion proportion.

6. The method according to any one of claims 1 to 5, characterized in that: If the QP values ​​of the consecutive n frames of data encoded before the current moment are all greater than the preset QP threshold, adjusting the image signal processing ISP image parameters includes: Based on the QP threshold, determining ISP target image parameters from the ISP image parameters, the ISP target image parameters including noise reduction parameters and sharpening parameters; increasing the level of the noise reduction parameter; Reduce the level of the sharpening parameter.

7. The method according to any one of claims 1 to 5, characterized in that: Before adjusting the image signal processing ISP image parameters, the method further includes: When the network signal weakens and the network bandwidth cannot meet the data volume of the bitstream transmission, the target bitrate is adjusted based on the network bandwidth at the current moment.

8. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: If the QP values ​​of the consecutive n frames of data encoded after adjusting the encoding reference relationship are all greater than the QP threshold, the encoding frame rate and resolution are reduced.

9. The method according to claim 1, characterized in that: The method further comprises: When the motion state indicates that the front-end device is stationary and it is detected that the current network bandwidth is greater than a preset first bandwidth threshold and less than a preset second bandwidth threshold, the encoding reference relationship is adjusted to a fourth encoding reference relationship for encoding, and the fourth encoding reference relationship includes: the first frame is encoded as an I frame, an I frame is encoded after each sixth preset interval, and between two I frames, the previous frame and the last I frame that has been encoded are referenced as reference frames and encoded as a p frame, and the first bandwidth threshold is less than the second bandwidth threshold.

10. The method according to claim 2, characterized in that The method further comprises: During encoding using the second encoding reference relationship, if the QP values ​​of the encoded n consecutive frames of data are all greater than the QP threshold and it is detected that the current network bandwidth cannot meet the data volume of the code stream transmission, the motion state of the front-end device is adjusted to still.

Images