Video encoding method and apparatus, device, and readable storage medium

Abstract

The present application discloses a video encoding method and apparatus, a device, and a readable storage medium. The method comprises: acquiring the current encoding bit rate of a video encoder; acquiring a frame sequence to be encoded of a target video source, and starting to encode an initial video frame in said frame sequence by means of the video encoder on the basis of the encoding bit rate; determining a first image texture value of a first video frame currently prepared to be encoded in said frame sequence, and determining a second image texture value of a second video frame immediately preceding the first video frame, the first video frame being any video frame following the initial video frame; when the first image texture value is greater than the second image texture value, determining an image texture ratio on the basis of the first image texture value and the second image texture value; and adjusting the encoding bit rate of the video encoder on the basis of the image texture ratio to obtain a target encoding bit rate, and encoding the first video frame on the basis of the target encoding bit rate. In this way, the bit rate of the video encoder is adjusted at the frame level, ensuring the quality of encoded video frames.

Description

Video encoding methods, apparatus, devices, and readable storage media

[0001] This application claims priority to Chinese Patent Application No. 202410913458X, filed on July 9, 2024, entitled "Video Coding Method, Apparatus, Device and Readable Storage Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of computer technology, specifically to video coding. Background Technology

[0003] With the continuous development of streaming media technology, the operation of streaming media services such as online live streaming, video downloading and playback, distance education, and telemedicine has been driven, enriching people's lives. For streaming media services involving video, in order to increase the transmitted video bandwidth under limited network bandwidth, video encoding is required during video stream data transmission. In the video encoding process, the video source bitrate needs to be determined first, so that the encoding bitrate of the video encoder can be determined based on the video source bitrate for video encoding.

[0004] When determining the video source bitrate, the relevant technology calculates the video source bitrate at fixed intervals and adjusts the encoding bitrate of the video encoder synchronously based on the determined video source bitrate.

[0005] In the process of researching and practicing related technologies, the inventors of this application discovered that related technologies statistically analyze video source bitrate over a fixed period of time. The statistically obtained video source bitrate is relatively fixed. After adjusting the encoding bitrate of the video encoder synchronously according to the determined video source bitrate, the video encoder will encode according to the fixed encoding bitrate during this period. Due to the diversity of video content, this will affect the video encoding quality and lead to a reduction in the image quality of the encoded video frames. Summary of the Invention

[0006] This application provides a video encoding method, apparatus, device, and readable storage medium, which can flexibly adjust the encoding bitrate of the video encoder according to the frame level to ensure the quality of the video frames after encoding.

[0007] To solve the above-mentioned technical problems, this application provides the following technical solution:

[0008] This application provides a video encoding method, including:

[0009] Get the current bitrate of the video encoder;

[0010] The sequence of frames to be encoded from the target video source is obtained, and the first video frame in the sequence is encoded by the video encoder according to the encoding bitrate.

[0011] Determine the first image texture value of the first video frame currently to be encoded in the frame sequence to be encoded, and determine the second image texture value of the second video frame adjacent to the first video frame, wherein the first video frame is any video frame after the first video frame;

[0012] When the texture complexity identified by the first image texture value is greater than the texture complexity identified by the second image texture value, the image texture ratio is determined based on the first image texture value and the second image texture value.

[0013] The encoding bitrate of the video encoder is adjusted according to the image texture ratio to obtain the target encoding bitrate, and the first video frame is encoded according to the target encoding bitrate.

[0014] Accordingly, embodiments of this application provide a video encoding apparatus, including:

[0015] The acquisition unit is used to acquire the current encoding bitrate of the video encoder.

[0016] The encoding unit is used to acquire the sequence of frames to be encoded from the target video source, and to encode the first video frame in the sequence of frames to be encoded by the video encoder according to the encoding bitrate.

[0017] The first determining unit is used to determine the first image texture value of the first video frame currently to be encoded in the frame sequence to be encoded, and to determine the second image texture value of the second video frame adjacent to the first video frame, wherein the first video frame is any video frame after the first video frame;

[0018] The second determining unit is used to determine an image texture ratio based on the first image texture value and the second image texture value when the texture complexity identified by the first image texture value is greater than the texture complexity identified by the second image texture value.

[0019] The adjustment unit is used to adjust the encoding bitrate of the video encoder according to the image texture ratio to obtain the target encoding bitrate, and to encode the first video frame according to the target encoding bitrate.

[0020] Furthermore, this application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described video encoding method.

[0021] In another aspect, embodiments of this application provide a storage medium for storing a computer program for executing the methods described above.

[0022] In another aspect, embodiments of this application provide a computer program product including a computer program, which, when run on a computer, causes the computer to perform the methods described above.

[0023] This application embodiment can obtain the current encoding bitrate of the video encoder; obtain the sequence of frames to be encoded from the target video source, and start encoding the first video frame in the sequence of frames to be encoded according to the encoding bitrate using the video encoder; determine the first image texture value of the first video frame currently to be encoded in the sequence of frames to be encoded, and determine the second image texture value of the second video frame adjacent to the first video frame, wherein the first video frame is any video frame after the first video frame; when the first image texture value is greater than the second image texture value, determine the image texture ratio based on the first image texture value and the second image texture value; adjust the encoding bitrate of the video encoder according to the image texture ratio to obtain the target encoding bitrate, and encode the first video frame according to the target encoding bitrate.

[0024] Based on this, this application can obtain the current encoding bitrate of the video encoder in real time, and then encode the video frames in the current frame sequence of the target video source according to the encoding bitrate. Furthermore, during the encoding process, the texture complexity between the first video frame to be encoded and the adjacent second video frame is compared in real time. Texture complexity represents the richness of the video frame's content to evaluate the required encoding bitrate. If the texture of the first video frame is more complex than the previous frame, the image texture ratio between the first and second video frames is determined based on the first and second image texture values. Finally, the encoding bitrate of the video encoder is adjusted to the target bitrate according to the image texture ratio. The target encoding bitrate is set so that the video encoder encodes the first video frame according to the adjusted target encoding bitrate. In this way, compared with the related technology where the video encoder encodes video frames according to the video source bitrate counted over a fixed duration, this application can compare the texture complexity between the preceding and following video frames during the encoding process of the target video source's frame sequence. When the texture of the following video frame is more complex than that of the preceding video frame, the encoding bitrate of the video encoder is adjusted according to the texture ratio between the preceding and following video frames. In this way, the encoding bitrate of the video encoder can be flexibly and finely adjusted at the frame level to ensure the encoding quality of the following video frames and guarantee the video frame image quality of the video encoding.

[0025] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0026] Figure 1 is a schematic diagram of a video encoding system provided in an embodiment of this application;

[0027] Figure 2 is a flowchart illustrating the steps of the video encoding method provided in an embodiment of this application.

[0028] Figure 3 is an example of an image texture value calculation scenario in the video encoding method provided in the embodiments of this application;

[0029] Figure 4 is a schematic flowchart of another step of the video encoding method provided in the embodiment of this application;

[0030] Figure 5 is an example diagram of a video encoding scenario provided in an embodiment of this application;

[0031] Figure 6 is an example diagram of bitrate variation in a code control scenario for video encoding provided in the embodiments of this application;

[0032] Figure 7 is an example diagram of the video encoding process provided in an embodiment of this application;

[0033] Figure 8 is an example diagram of the calculation scenario of image texture value of video frame provided in the embodiment of this application;

[0034] Figure 9 is a schematic diagram of the video encoding device provided in an embodiment of this application;

[0035] Figure 10 is a schematic diagram of the structure of the terminal provided in an embodiment of this application;

[0036] Figure 11 is a schematic diagram of the server structure provided in an embodiment of this application. Detailed Implementation

[0037] To enable those skilled in the art to better understand the solutions of this application, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] It is understood that in the specific implementation of this application, data related to target video sources is involved. When the above embodiments of this application are applied to specific products or technologies, permission or consent from the target is required, and the collection, use and processing of related data must comply with relevant laws, regulations and standards.

[0039] Furthermore, when this application embodiment needs to obtain target video source and related data, it will obtain separate permission or separate consent for the target video source and related data through pop-up windows or redirection to a confirmation page. After clearly obtaining separate permission or separate consent for the target video source and related data, it will then obtain the necessary target video source and related data for enabling this application embodiment to operate normally.

[0040] It should be noted that while some processes described in the specification, claims, and accompanying drawings contain multiple steps that appear in a specific order, it should be clearly understood that these steps may not be performed in the order they appear herein, or may be performed in parallel. The step numbers are merely used to distinguish different steps and do not represent any particular order of execution. Furthermore, descriptions such as "first," "second," or "objective" in this document are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0041] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0042] Before providing a further detailed description of the embodiments of this disclosure, the terms and concepts used in these embodiments are explained, and they are subject to the following interpretations:

[0043] IDR frames: In video coding algorithms (H.264 / H.265 / H.266 / AV1, etc.), images (i.e., video frames) are organized as sequences. The first image in a sequence is an Instantaneous Decoding Refresh (IDR) frame. IDR images are all intrapicture (I) frames, which can also be understood as keyframes.

[0044] I-frame: Internal image, keyframe. An IDR frame will clear the Decoded Picture Buffer (DPB) reference frame list, while an I-frame will not. An IDR image is always an I-image, but an I-image is not necessarily an IDR image. A sequence can contain many I-images, and image frames following an I-image frame can reference image frames between two I-image frames as motion references.

[0045] Code control: Encoder bitrate control, specifically refers to the allocation of bits and adjustment of quantization parameters (QP) for the frame to be encoded by utilizing the existing information of the encoder, the actual number of bits generated after encoding, and the state of the buffer, so as to ensure that optimal video quality is achieved through reasonable bit allocation under a limited target bitrate.

[0046] Bitrate / bitstream: This refers to the amount of data a video file uses per unit of time. Also called bitrate or bitstream rate, it can be simply understood as the sampling rate. It's the most important part of image quality control in video encoding, and is generally measured in kilobits per second (kbps) or megabits per second (mbps). At the same resolution, the higher the bitstream of a video file, the lower the compression ratio and the higher the image quality. A higher bitstream indicates a higher sampling rate per unit of time, higher data stream precision, and a file that is closer to the original file after encoding. This results in better image quality, clearer picture, and higher requirements for the decoding capabilities of the playback device.

[0047] CQP: Constant Quantization Parameter.

[0048] QoS: Quality of Service, is mainly responsible for managing services and providing service differentiation from the network perspective. Network entities handle different services according to different quality requirements. Common monitoring indicators include stuttering rate, stuttering time, and first frame.

[0049] QoE: Quality of Experience. The evaluation subject is the end user, and the evaluation objects are the business and the network supporting the business. Taking the video field as an example, the evaluation indicators are not limited to the average user video viewing time, video quality and clarity, screen latency and buffering, etc. These indicators will affect the viewing experience (QoE) on the viewer's end, and thus also affect the advertising conversion effect.

[0050] GOP stands for group of pictures, specifically referring to the interval between two I-frames, such as interval duration and interval length. The Instant Decode Refresh (IDR) frame is the first I-frame in a GOP, marking the start of a new sequence encoding for the video stream. Its purpose is to immediately refresh the decoder, thus preventing prediction errors from propagating.

[0051] This application provides a video encoding method, apparatus, device, and readable storage medium. Specifically, this application will describe the video encoding apparatus from the perspective of the video encoding apparatus, which can be integrated into a computer device. The computer device can be a server or a target terminal, etc. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The target terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, smart home appliance, vehicle terminal, smart voice interaction device, aircraft, etc., but is not limited to these.

[0052] This application provides a video encoding method that uses the interval between keyframes in a video frame sequence to calculate the video source bitrate, which is then used to update the encoding bitrate of the video encoder. Based on this, the encoding bitrate of the video encoder is adjusted at the frame level by comparing the texture complexity between consecutive frames, and the video frames are encoded according to the adjusted bitrate. The specific implementation is illustrated in the following examples:

[0053] Related technologies determine the video source bitrate by calculating it over a fixed period or by using bitrate information carried in the original video. This bitrate is then used to update the video encoder's bitrate, allowing the encoder to encode video frames in the video source sequence according to the bitrate. However, this method of calculating the video source bitrate over a fixed period and updating the encoder's bitrate results in a fixed bitrate for a certain time. If the encoder encodes video frames according to this fixed bitrate, it may lead to poor image quality in video frames with rich image content and complex textures after encoding, reducing the overall video quality and affecting the viewing experience.

[0054] To address the aforementioned issues, this application proposes a video encoding method. During the encoding process of a sequence of frames to be encoded from a target video source, the method compares the texture complexity between consecutive video frames. When the texture of a later video frame is more complex than that of a previous video frame, the encoding bitrate of the video encoder is adjusted according to the ratio of texture complexity between the two frames. This allows for flexible and precise adjustment of the video encoder's bitrate at the frame level, ensuring the encoding quality of later video frames and guaranteeing the image quality of the encoded video frames. Please refer to the following specific embodiments for details.

[0055] This application provides a video encoding system. The devices in this system may include a server and / or a terminal. The terminal may request the server to execute the video encoding method of this application.

[0056] For example, the system includes a server or terminal. The terminal or server can obtain the current encoding bitrate of the video encoder; obtain the sequence of frames to be encoded from the target video source, and start encoding the first video frame in the sequence of frames to be encoded according to the encoding bitrate using the video encoder; determine the first image texture value of the first video frame currently to be encoded in the sequence of frames to be encoded, and determine the second image texture value of the second video frame adjacent to the first video frame, wherein the first video frame is any video frame after the first video frame; when the first image texture value is greater than the second image texture value, determine the image texture ratio based on the first image texture value and the second image texture value; adjust the encoding bitrate of the video encoder according to the image texture ratio to obtain the target encoding bitrate, and encode the first video frame according to the target encoding bitrate.

[0057] For example, referring to Figure 1, which is a schematic diagram of a video encoding system provided in an embodiment of this application, the system includes a server and a terminal. The terminal is equipped with a client, which can send or obtain real-time video streaming data to or from the server backend, such as compressing and storing object attribute data in various business areas, querying object attribute data, etc.

[0058] The terminal can directly send or receive real-time video stream data to the server; in addition, a video client can be installed on the terminal, which can be used to send or receive real-time video stream data to the server.

[0059] In the process of executing the video encoding method, the server can obtain the current encoding bitrate of the video encoder; obtain the sequence of frames to be encoded from the target video source, and start encoding the first video frame in the sequence of frames to be encoded according to the encoding bitrate using the video encoder; determine the first image texture value of the first video frame currently to be encoded in the sequence of frames to be encoded, and determine the second image texture value of the second video frame adjacent to the first video frame, wherein the first video frame is any video frame after the first video frame; when the first image texture value is greater than the second image texture value, determine the image texture ratio based on the first image texture value and the second image texture value; adjust the encoding bitrate of the video encoder according to the image texture ratio to obtain the target encoding bitrate, and encode the first video frame according to the target encoding bitrate.

[0060] It should be noted that the video encoding system is applicable to both live video streaming and video-on-demand scenarios. In live video streaming, the terminal can include both a broadcaster and a viewer. The broadcaster sends video source data to the server in real time. The server encodes the video frames from the source and sends the encoded video frame stream to the viewer for decoding and playback. In video-on-demand, the terminal can be understood as either the viewer or the on-demand client. The server loads video data directly from pre-stored storage or other servers, using the pre-stored or loaded video as the video source. The server encodes the video frames from the source's video frame stream sequence and sends the encoded video frame stream to the terminal for decoding and playback.

[0061] For example, taking live video streaming as an example, suppose the broadcaster is streaming a game. The game footage is the target video source, and the broadcaster transmits the game footage to the server in real time as a stream. This stream data is transmitted to the server in sequence. The server determines the video source encoding based on the interval between keyframes in the sequence. Specifically, it determines the encoding in real time based on the granularity of the group of pictures (GOP) between two keyframes (I-frames). The frame interval can represent a group of video frames (the group of frames to be encoded). Specifically, the frame length (i.e., the interval) between two keyframes (I-frames) and the frame rate of the video source can be determined first. Then, the number of video frame groups per unit time, i.e., the video source bitrate, can be determined based on the ratio between the frame rate and the frame length. Then, the video encoding kernel (i.e., the video encoder) is dynamically loaded and updated according to the current video source bitrate so that the video encoder can obtain the current basic encoding bitrate.

[0062] Furthermore, for the video frames in the frame sequence to be encoded, the server begins encoding the video frames in the frame sequence according to the encoding bitrate of the video encoder. During the encoding process, the complexity of the video frames between adjacent frames is determined. For example, the complexity can be represented by image texture. First, the image texture value of the current frame and the image texture value of the previous frame are determined. By comparing the image texture values ​​between the previous and next frames, it is determined whether the current frame is more complex than the previous frame. If the image texture value of the video frame to be encoded is greater than the image texture value of the previous adjacent video frame, it means that the current video frame is more complex than the previous video frame. At this time, the image texture ratio between the current video frame and the previous video frame can be determined. Since the image texture ratio is greater than 1, the encoding bitrate in the video encoder is adjusted according to the image texture ratio. The target encoding bitrate obtained by the adjustment is greater than the encoding bitrate when the previous frame was encoded. The current video frame is encoded according to the target encoding bitrate to ensure the image encoding quality of the current more complex video frame and to guarantee the video frame image quality. Afterwards, the encoded video frame data is transmitted to the viewer for decoding and playback. The viewer can then display the rich and complex game scenes in a better picture quality, ensuring reliability.

[0063] It should be noted that for real-time audio and video scenarios, such as instant messaging scenarios involving voice and video calls, the above video live streaming scenarios can be referenced. Their real-time processes are the same or similar, and will not be elaborated on here.

[0064] For example, taking video on demand (VOD) as an example, VOD requires real-time video streaming data. The server directly loads video from pre-stored data or from other servers, using the pre-stored or loaded video as the video source. The server determines the video source encoding based on the interval between keyframes in the video frame stream sequence of the video source. Specifically, it determines the encoding in real time based on the granularity of the frame interval (group of pictures, GOP) between two keyframes (I-frames). The frame interval can represent a group of video frames (a group of frames to be encoded). Specifically, the frame length between two keyframes (I-frames) and the frame rate of the playback client can be determined first. Then, the number of video frame groups per unit time, i.e., the video source bitrate, can be determined based on the ratio between the frame rate and the frame length. Subsequently, the video encoding kernel (i.e., the video encoder) is dynamically loaded and updated according to the current video source bitrate so that the video encoder can obtain the current basic encoding bitrate.

[0065] Furthermore, for the video frames in the frame sequence to be encoded, the server begins encoding the video frames in the frame sequence according to the encoding bitrate of the video encoder. During the encoding process, the complexity of the video frames between adjacent frames is determined. For example, the complexity can be represented by image texture. First, the image texture value of the current frame and the image texture value of the previous frame are determined. By comparing the image texture values ​​between the previous and next frames, it is determined whether the current frame is more complex than the previous frame. If the image texture value of the video frame to be encoded is greater than the image texture value of the previous adjacent video frame, it indicates that the current video frame is more complex than the previous video frame. At this time, the image texture ratio between the current video frame and the previous video frame can be determined. Since the image texture ratio is greater than 1, the encoding bitrate in the video encoder is adjusted according to the image texture ratio. The target encoding bitrate obtained by the adjustment is greater than the encoding bitrate when the previous frame was encoded. The current video frame is encoded according to the target encoding bitrate to ensure the image encoding quality of the current more complex video frame and to guarantee the video frame image quality. Afterward, the encoded video frame data is transmitted to the terminal (video client) for decoding and playback. In this way, the terminal can display rich and complex game scenes with better picture quality and reliability.

[0066] Based on the above examples, compared to the video encoder's scheme of encoding video frames according to the video source bitrate with a fixed duration in related technologies, this application can compare the texture complexity between consecutive video frames during the encoding process of the target video source's frame sequence. When the texture of the subsequent video frame is more complex than that of the previous video frame, the encoding bitrate of the video encoder is adjusted according to the texture ratio of the consecutive video frames. In this way, the encoding bitrate of the video encoder can be flexibly and finely adjusted at the frame level to ensure the encoding quality of the subsequent video frames and guarantee the video frame image quality of the video encoding.

[0067] It should be noted that the above are just examples and can be applied to other video encoding scenarios, which will not be elaborated here.

[0068] For ease of understanding, the steps of the video encoding method will be described in detail below. It should be noted that the order of the following embodiments is not intended to limit the preferred order of the embodiments.

[0069] In this embodiment, the description will focus on the video encoding device, which can be integrated into a computer device, such as a terminal or server. Referring to Figure 2, which is a flowchart illustrating the steps of the video encoding method provided in this embodiment, this embodiment takes the video encoding device being integrated into a server as an example. When the processor on the server executes the program instructions corresponding to the video encoding method, the specific process is as follows:

[0070] 101. Obtain the current bitrate of the video encoder.

[0071] Video encoding can be understood as compressing video source data and transmitting the compressed data to the playback device for decoding and playback, so that the corresponding video image is displayed on the playback page. It's important to note that video stream data is organized and transmitted in the form of a frame sequence. Video encoding involves encoding the video frames in the frame sequence using a video encoder. The video encoder has a bitrate; that is, when encoding the video frames in the frame sequence, the encoder encodes them according to the bitrate. This means that the video frames in the frame sequence are compressed sequentially according to the bitrate, so that the bitrate of each compressed frame is lower than the bitrate before compression, resulting in a video file corresponding to the sequence. This can be understood as video segments, allowing for transmission to the playback device with less communication bandwidth.

[0072] A video encoder is an application used to compress and convert video signals into digital formats. Its main function is to reduce the size of video files through compression technology, thus requiring less bandwidth and storage space during network transmission or storage. A video encoder can convert raw video signals into various digital formats, such as H.264, MPEG-4, AVC, and VP9. The functions of a video encoder include compressing video data, changing video formats, and adjusting image quality. It takes multiple input signals (such as HDMI, SDI, IP video, etc.) and encodes these signals into compressed digital formats, such as H.264 or H.265, which require less bandwidth for transmission and storage. Further, the data is transmitted over the network to a backend monitoring center, where it is decoded by corresponding decoding software or hardware running on the terminal.

[0073] However, the content of different video frames may differ, which can be understood as varying levels of content richness or complexity. Video frames with richer content typically require more memory than those with simpler content. Therefore, during encoding at a uniform bitrate, simpler video frames tend to have better image quality after encoding, while more content-rich frames tend to have lower image quality. To address this, the bitrate of the video encoder can be adjusted to reduce the lower image quality of more content-rich frames after encoding.

[0074] The encoding bitrate can be defined as the number of bits of information transmitted per second, and can be expressed in kilobits per second (kbps) or megabits per second (mbps). Specifically, the bitrate is the number of bits of data or data throughput transmitted per unit of time, reflecting the transmission speed and quality of video data. During video encoding, the encoding bitrate can be controlled to optimize video quality and file size.

[0075] To achieve a reasonable bitrate for the video encoder, the bitrate can be adjusted based on the source video bitrate. Related technologies periodically acquire the source video bitrate for fixed durations (e.g., 1 second, 3 seconds, etc.) and adjust the encoder bitrate accordingly. However, adjusting the bitrate based on a fixed-duration bitrate results in a fixed bitrate with coarse granularity. Since real-time video sequences contain multiple frames with varying complexity, using a coarse bitrate can cause more complex frames to become blurry and of poor quality after encoding.

[0076] The video bitrate refers to the number of bits transmitted per second in the source video, expressed in kilobits per second (kbps) or megabits per second (mbps). A higher bitrate indicates a larger amount of data transmitted per unit of time, resulting in higher video precision and quality, and a clearer picture. Bitrate is one of the important factors determining video file size and image quality; at the same resolution, a higher bitrate results in a larger video file size and better image quality.

[0077] In this embodiment of the application, in order to achieve better image quality after encoding of video frames with high complexity, the acquisition granularity of the video source bitrate is reduced to acquire the video source bitrate of the target video source, so as to synchronously adjust the encoding bitrate of the video encoder, thereby achieving a more precise adjustment of the encoding bitrate of the video encoder, which serves as the base bitrate for video encoding and has reliability.

[0078] The target video source can be a real-time transmitted video source, which can be understood as the video signal source or video source end, providing real-time streaming data. For example, it could be the streaming data of video frames uploaded and transmitted from the broadcaster's end to the server in a live video broadcast (i.e., the signal source); it could also be the streaming data of video frames read and loaded in real-time from other platforms or databases by the server in a video-on-demand scenario; or it could be the audio and video streaming data in an audio-visual call scenario, and so on. These are just examples and not intended as specific limitations; other video sources capable of real-time streaming data are also applicable, but not all are listed here.

[0079] In some implementations, the video source bitrate can be determined based on the interval between key video frames in the target video source, so that the encoding bitrate of the video encoder can be updated synchronously according to the video source bitrate. For example, step 101 may include: obtaining the current video source bitrate of the target video source; and updating the encoding bitrate of the video encoder synchronously according to the video source bitrate.

[0080] The encoding bitrate of the video encoder is adjustable. The real-time encoding bitrate of the video encoder can be adjusted based on the interval between two adjacent key video frames of the target video source. The video frame stream data of the target video source can be organized and transmitted to the server in the form of a video frame sequence. This video frame sequence can include different types of video frames, such as key video frames (Intra-coded Frames, I-frames), predicted frames (P-frames), and bi-directional predicted frames (B-frames). The video source bitrate between two key frames can be statistically determined based on the interval between them (group of pictures, GOP). Specifically, the frame rate of the target video source is determined, and the interval between two adjacent key video frames in the transmission sequence of the target video source is detected in real time, such as the interval length or interval duration. Then, the video source bitrate is calculated based on the interval between the two adjacent key video frames and the frame rate, for example, by determining the video source bitrate based on the ratio between the frame rate and the interval. Furthermore, the encoding bitrate in the video encoder is updated synchronously based on the video source bitrate, ensuring that the current encoding bitrate is compatible with the sequence of frames to be encoded between the current two keyframes in the video frame sequence—essentially, a subsequence or segment sequence. Based on this, since the real-time video frame stream data of the target video source is organized in sequence, the real-time video source bitrate is determined by using the interval between keyframes. This bitrate serves as the base bitrate for the video encoder, better meeting the encoding bitrate requirements of each video frame sequence, offering both flexibility and reliability.

[0081] In this embodiment, the encoding bitrate can be real-time. The current encoding bitrate of the video encoder can be the initial encoding bitrate that is synchronously updated based on the video source bitrate, or it can be the historical encoding bitrate used when encoding the previous video frame during the video encoding process, depending on the actual situation. It should be noted that the "previous video frame" can refer to "a video frame in the sequence of frames to be encoded" in this embodiment. The "sequence of frames to be encoded" will be described in detail later and will not be discussed here.

[0082] By using the above methods, the real-time encoding bitrate of the video encoder can be obtained, which can be used as the base bitrate for video encoding to better match the encoding bitrate requirements of each video frame sequence, and also has reliability.

[0083] 102. Obtain the sequence of frames to be encoded from the target video source, and start encoding the first video frame in the sequence of frames to be encoded using a video encoder according to the encoding bitrate.

[0084] In this embodiment, after obtaining the real-time encoding bitrate of the video encoder, the sequence of frames to be encoded corresponding to the target video source can be obtained. The video encoder can then start encoding the video frames in the sequence of frames to be encoded according to the current encoding bitrate. Since the encoding process of the video encoder can be frame-by-frame, the first video frame in the sequence of frames to be encoded can be encoded according to the encoding bitrate to compress the video frames, increase the data transmission rate per unit time under a specific bandwidth, and ensure the smoothness of video playback at the playback end.

[0085] The frame sequence to be encoded can be a sub-sequence or a sequence segment transmitted in real time by the target video source. Specifically, all video frames provided by the target video source are transmitted as a video frame sequence. The frame sequence to be encoded can be a sub-sequence of the entire video frame sequence. For example, it can be a sequence consisting of two currently adjacent key video frames and non-key video frames located between these two key video frames (such as the aforementioned forward prediction frames and / or bidirectional prediction frames), or it can be a sequence consisting of multiple key video frames and non-key video frames. There is no limitation here.

[0086] Here, the first video frame can refer to the first video frame in the current sequence of frames to be encoded. It should be noted that this first video frame refers to the first frame in the sub-sequence currently being transmitted by the target video source; that is, the first frame is different at different times during the video encoding process, depending on the real-time sequence of frames to be encoded. Therefore, the first video frame can be the first frame of the entire video frame sequence corresponding to the target video source, or it can be a frame in the middle of the video frame sequence; there is no limitation here. The type of the first video frame can be a key video frame.

[0087] In this embodiment of the application, the server, when performing video encoding, can simultaneously receive and acquire video frame stream data from the target video source and encode the real-time acquired video frame stream data. Assuming the video frame stream data is transmitted in sequence, the server can sequentially receive and acquire each sequence of frames to be encoded from the target video source and encode the video frames in each sequence sequentially. Therefore, the encoding process of the target video source's video stream data is performed in real-time. If the server has already encoded the video frames in historical sequences of frames to be encoded, when the server acquires a sequence of frames to be encoded, and the video encoder in the server has been updated and adjusted to the corresponding encoding bitrate, the video encoder uses the current encoding bitrate as the base bitrate, that is, it starts encoding the first video frame in the sequence of frames to be encoded according to the current real-time encoding bitrate.

[0088] For example, taking live video streaming as an example, the target video source can be understood as the live video source of the broadcaster. The server obtains the live frame sequence from the broadcaster in real time. After the video source bitrate calculated based on the interval between key video frames in the live frame sequence is synchronously updated to the encoding bitrate of the video encoder, the video encoder uses the current real-time encoding bitrate as the basis and starts to encode the first live frame in the frame sequence to be encoded according to the encoding bitrate to obtain encoded stream data for subsequent transmission to the viewer.

[0089] For example, taking video-on-demand as an example, the target video source can be understood as the target video stream data loaded in real time from a video database or other server. This real-time loaded target video stream data is transmitted to the server in sequence. The server obtains the sequence of frames to be encoded from the target video stream data in real time. After the video source bitrate calculated based on the interval between key video frames is synchronously updated to the encoding bitrate of the video encoder, the video encoder uses the current real-time encoding bitrate as a basis and starts encoding the first target video frame in the sequence of frames to be encoded according to the encoding bitrate to obtain the encoded stream data for subsequent transmission to the playback end.

[0090] By using the above methods, the sequence of frames to be encoded corresponding to the target video source can be obtained. The current encoding bitrate of the video encoder is used as the base bitrate. The video encoder starts encoding the video frames in the sequence of frames to be encoded according to the current encoding bitrate, thereby compressing the video frames, increasing the data transmission rate per unit time under a specific bandwidth, and ensuring the smoothness of video playback on the playback end.

[0091] 103. Determine the first image texture value of the first video frame currently to be encoded in the frame sequence to be encoded, and determine the second image texture value of the second video frame adjacent to the first video frame.

[0092] In the embodiments of this application, during the encoding process of video frames in the sequence of frames to be encoded, the complexity of the video frame corresponding to the current encoding progress and the complexity of the previous video frame can be determined. Thus, the complexity comparison between the current video frame and the previous video frame can be used to determine whether the video frame corresponding to the current encoding progress is more complex than the previous video frame, thereby deciding whether to adjust the encoding bitrate of the video encoder, which has reliability.

[0093] It should be noted that, in representing the complexity of video frames in this application embodiment, it can be described through the constituent elements and / or grayscale changes of the video frames. For example, it can be described using algorithms such as Euclidean distance, statistical histograms, Local Binary Patterns (LBP) detection algorithms, and Convolutional Neural Networks (CNN) feature extraction and classification. For ease of understanding, this application embodiment will use a local binary pattern detection algorithm to represent the complexity of preceding and following video frames. This local binary pattern can refer to image texture to represent the complexity of the video frame.

[0094] In this sequence, the first video frame can be the video frame ready to be encoded corresponding to the current encoding progress in the frame sequence to be encoded, while the second video frame can be the video frame preceding the first video frame currently ready to be encoded in the frame sequence to be encoded. It should be noted that the second video frame is usually the video frame currently being encoded or has already been encoded, while the first video frame refers to the next video frame to be encoded. Therefore, the first video frame can be any video frame after the first video frame in the frame sequence to be encoded. Assuming there are N video frames in the frame sequence to be encoded in a temporal arrangement, the first video frame can be the i-th video frame, and the second video frame can be the (i-1)-th video frame, where i is greater than 1 and less than or equal to N.

[0095] The image texture value can be a parameter representing the texture features of a video frame, used to quantitatively or qualitatively describe the texture characteristics of the video frame. The image texture value can be determined based on the analysis of texture features using important surface grayscale information in the video frame. It should be noted that the texture feature is a value calculated from the video frame, used to quantify the characteristics of grayscale level changes within a region. Specifically, it is generated by calculating the grayscale co-occurrence matrix of the sub-image formed by each small window and the texture feature value. Therefore, the image texture value can not only reflect the visual complexity of an image but also serve as an important indicator for quantifying the complexity of image information. Thus, the complexity of image information in the first video frame can be represented by the first image texture value, and the complexity of image information in the second video frame can be represented by the second image texture value. It should be noted that the image texture value can be a single numerical value or a matrix form, such as an image texture value represented as an image matrix containing three two-dimensional matrices (R, G, B), with values ​​between 0 and 255, representing the depth of color.

[0096] In some implementations, the corresponding first image texture value can be determined based on the grayscale value of each first pixel in the first video frame. For example, "determining the first image texture value of the first video frame currently to be encoded in the frame sequence to be encoded" in step 103 may include:

[0097] (103.1) Obtain the grayscale value of each first pixel in the first video frame currently to be encoded in the frame sequence to be encoded;

[0098] (103.2) Determine the first image texture value of the first video frame based on the gray value of each first pixel.

[0099] The grayscale value can be the color depth of a pixel in a black and white image, that is, the color depth of each first pixel in the first video frame. The grayscale value ranges from 0 to 255, with 255 for white and 0 for black. The grayscale value represents the change in brightness from dark to light, corresponding to the color in the video frame from black to white. It can reflect the statistical characteristics and contrast of the video frame. It should be noted that the color value of each pixel in a grayscale image can be understood as grayscale, which refers to the color depth of a point in a black and white image.

[0100] Specifically, during the encoding process of video frames in the frame sequence to be encoded, in order to determine whether the first video frame is more complex than the previous video frame (i.e., the second video frame), it is necessary to first determine the image texture values ​​of the first and second video frames respectively. Taking the calculation of the first image texture value of the first video frame as an example, for the first video frame in the frame sequence to be encoded where the current encoding progress is to be prepared for encoding, the grayscale value corresponding to each first pixel in the first video frame can be obtained. For example, the grayscale value of each first pixel can be obtained by weighted averaging of the color values ​​of the three color channels (red, green, and blue) to represent the contribution of the color values ​​of the three color channels to the brightness; or, the color value of each first pixel in the three color channels can be determined, and the maximum / minimum value of the color values ​​of the first pixel in the three color channels can be used as the grayscale value of the first pixel; the above grayscale value determination process is only an example and is not intended as a specific limitation. Further, the first image texture value of the first video frame is determined based on the grayscale value of each first pixel.

[0101] Similarly, the second image texture value of the second video frame can be determined in the same way as above. Based on this, the relationship between the first and second image texture values ​​can be used to determine whether the first video frame is more complex than the previous video frame (i.e., the second video frame), so as to adjust the bitrate for video encoding.

[0102] It should be noted that when calculating and determining the first image texture value of the first video frame, the first image texture value can be generated not only based on all area information of the first video frame, but also based on the grayscale values ​​of the target area information, target object information and / or target object information in the first video frame; similarly, the second image texture value can be determined based on the grayscale values ​​of the target area information, target object information and / or target object information in the second video frame.

[0103] For example, when determining the image texture values ​​of the first video frame and the second video frame, the image texture values ​​can be calculated based on the grayscale values ​​of pixels within the target image region in the video frame. The target image region can be the image region corresponding to a specific object or the image region of a fixed area.

[0104] Specifically, taking the calculation of the first image texture value of the first video frame as an example, a target mask image is obtained. The target mask image is an image that displays the pixels of the target image region in the first video frame. The first video frame is masked according to the target mask image to obtain the processed target first video frame. The target first video frame only displays the content within the target image region. Then, the gray value of each first pixel in the target first video frame is obtained so as to calculate the first image texture value of the target first video frame based on the gray value of each first pixel. Since the visualization of the target image region in the first video frame displays the corresponding region content information from the original first video frame, while the visualization of other image regions outside the target image region in the first video frame is blank, image texture value calculation based on this can specifically describe the key texture features of the target image region in the video frame, improving the computational efficiency of texture complexity. Subsequently, in the complexity comparison between the first and second video frames, for image frames such as forward prediction video frames or bidirectional prediction video frames that have removed redundant image information, if the only difference between the first and second video frames is the image content information in the target image region, this calculation of image texture value according to the mask method is particularly meaningful. It enables the rapid location of the content difference region between the first and second video frames, and uses the explicit difference information of this content difference region as the main basis for image texture value calculation, reducing the computational load in the image texture value calculation process and ensuring reliability.

[0105] In some implementations, the local texture value of each video frame sub-region can be determined first based on the grayscale value of the first pixel within each video frame sub-region in the first video frame, and then the first image texture value of the first video frame can be determined by combining the local texture values ​​of each video frame sub-region. For example, step (103.2) may include:

[0106] (103.2.1) Obtain the target local texture window;

[0107] (103.2.2) According to the target local texture window, the first pixel in the first video frame is divided to obtain multiple target local regions;

[0108] (103.2.3) Based on the grayscale value of the first pixel contained in each target local region, determine the first local texture value corresponding to each target local region in the first video frame;

[0109] (103.2.4) Determine the first image texture value of the first video frame based on the first local texture value corresponding to each target local region.

[0110] The target local texture window can be understood as a neighborhood region of a specific size, such as a region of 3 pixels by 3 pixels. Specifically, the target local texture window can be understood as the coverage area of ​​the operator in the local binary mode algorithm. In the target local texture window, the gray value of the first pixel located at the center of the window is used as the gray value judgment threshold of the current target local texture window. The gray values ​​of adjacent pixels are compared with the threshold. If the gray value of the surrounding pixels is greater than the gray value of the first pixel at the center, the position of the pixel is marked or assigned a value of 1; otherwise, it is assigned a value of 0.

[0111] The target local region can be a sub-region of an image in the first video frame, i.e., a sub-region of the video frame. It can be understood that the first video frame can be divided into multiple target local regions by dividing the first pixels according to the size of the target local texture window. The size of each target local region is the same as the size of the target local texture window. For example, taking the target local region of the first video frame as an example, if the target local texture window is a 3-pixel by 3-pixel range window, then each target local region consists of 3 first pixels in both length and width, meaning one target local region includes 9 first pixels. Alternatively, if the target local texture window is a 4-pixel by 4-pixel range window, one target local region contains 16 first pixels; and if it is a 5-pixel by 5-pixel range window, the target local region contains 25 first pixels.

[0112] Specifically, to determine the first image texture value of the first video frame, it can be calculated based on the first local texture values ​​of multiple regions within the first video frame. Specifically, a target local texture window can be obtained first, and the first video frame can be divided according to the size of the target local texture window, thus dividing the first video frame into multiple target local regions of uniform size. Further, after determining each target local region in the first video frame, for each target local region, the first local texture value corresponding to that target local region can be determined based on the grayscale value of the first pixel contained within that target local region. This first local texture value can be understood as the texture feature value of the first pixel at the center position of the target local region. Therefore, the first image texture value of the first video frame is obtained by summarizing the first local texture values ​​corresponding to each target local region. The calculation process can be a weighted calculation or a concatenation method; no limitation is made here. Therefore, the local texture value of each video frame sub-region can be determined based on the grayscale value of the first pixel in each video frame sub-region in the first video frame, and the first image texture value of the first video frame can be determined by combining the local texture values ​​of each video frame sub-region, so as to be used for the comparison of complexity between the first video frame and the second video frame in the future, thereby improving the accuracy of video frame complexity comparison and having reliability.

[0113] In some implementations, for each target local region in the first video frame, the difference between the grayscale value of the first pixel at the center of the region and the grayscale values ​​of the surrounding first pixels is determined, and a binarization transformation is performed based on the difference to obtain the first local texture value corresponding to each target local region. For example, step (103.2.4) may include: determining the central first pixel and the edge first pixels surrounding the central first pixel within each target local region; comparing the grayscale value of each edge first pixel with the grayscale value of the central first pixel for each target local region to obtain multiple comparison results for each target local region; and performing binarization processing on the multiple comparison results for each target local region to obtain the first local texture value corresponding to each target local region.

[0114] In this context, the central first pixel can be the pixel located at the center of the corresponding target local region in the first video frame, while the edge first pixels are the other pixels within the corresponding target local region that are located around the central first pixel. For example, assuming the target local region is the image region corresponding to a target local texture window of size 3 pixels by 3 pixels, then the target local region contains 9 first pixels. The pixel located at the center is the central first pixel, and the other 8 pixels located around the central first pixel are all edge first pixels. It should be noted that the central first pixel and the edge first pixel are considered in relation to a single target local region. Each target local region in the first video frame has its own central first pixel and edge first pixels.

[0115] Specifically, after dividing the first video frame into multiple target local regions, in order to obtain the first local texture value corresponding to the target local region, firstly, the first pixel located at the center of the target local region can be determined as the central first pixel, and other first pixels located around the central first pixel can be determined as edge first pixels; then, the gray value of the central first pixel is used as the gray value reference value or gray value threshold, and the gray value of each edge first pixel is compared with the gray value reference value of the center to obtain the gray value comparison result of each edge first pixel relative to the central first pixel; finally, based on the multiple gray value comparison results, a binarization transformation is performed to obtain the first local texture value of the target local region.

[0116] For example, if each target local region is an image region corresponding to a target local texture window of size 3 pixels by 3 pixels, then each target local region contains 9 first pixels: 1 central first pixel and 8 edge first pixels. Referring to Figure 3, taking one target local region as an example, the grayscale value of the central first pixel is 83, and the grayscale values ​​of the surrounding 8 edge first pixels are 44, 118, 192, 204, 250, 174, 61, and 32 respectively. For each edge first pixel, the grayscale value is compared with the grayscale value of the central first pixel, resulting in 8 grayscale comparison results. Each grayscale comparison result is then binarized. If the grayscale value of the edge first pixel is greater than that of the central first pixel, it is assigned a value of 1 after binarization; otherwise, it is not binarized. If the grayscale value is 0, then the value is assigned to 0. Therefore, for the above 8 grayscale comparison results, after binarization conversion, the values ​​are assigned to 0, 1, 1, 1, 1, 1, 0, 0 (clockwise order), which is represented by the 8-bit binary number 01111100. The image texture value is represented by 8 bits. For the first image texture value of each target local area, there can be 256 different results. The first image texture value of the target local area in binary form is converted to decimal form and represented as 124. Both the binary number "01111100" and the decimal number "124" can represent the first image texture value corresponding to the target local area.

[0117] It should be noted that for target local regions divided according to target local texture windows of other sizes, one or more pixels located in the middle of the target local region can be identified as the central pixel combination area. This central pixel combination area can be regarded as a whole, i.e., as the "central pixel". For example, taking the target local region of the first video frame as an example, the central first pixel combination area contains multiple first pixels located in the center of the target local region, and other first pixels in the target local region located outside the central first pixel combination area are regarded as edge first pixels. When calculating the image texture value of a target local region of this size, the gray value of the central first pixel combination region can be determined based on the gray values ​​of multiple pixels within the central first pixel combination region, or defined as the "target gray value of the central first pixel combination region". A weighted average of the gray values ​​of multiple first pixels within the central first pixel combination region is then performed to obtain the target gray value corresponding to the central first pixel combination region. Subsequently, the gray value of each edge pixel is compared with the target gray value corresponding to the central pixel combination region to obtain multiple gray-level comparison results. Finally, by binarizing these multiple gray-level comparison results, the first local texture value corresponding to the target local region is obtained.

[0118] For example, if each target local region is an image region corresponding to a target local texture window of size 4 pixels by 4 pixels, then the region contains 16 first pixels. When determining the central first pixel combination area within the target local region, the four first pixels in the "2 pixels by 2 pixels" area located at the center of the target local region can be taken as the central first pixel combination area. Then, based on the gray values ​​of the four first pixels in the central first pixel combination area, the target gray value of the central first pixel combination area is determined. For example, by taking a weighted average of the gray values ​​of these four first pixels, or by taking the maximum or minimum value among the gray values ​​of these four first pixels, as the target gray value of the central first pixel combination area. Subsequently, the 12 edge first pixels in the target local region located around the central first pixel combination area are compared with the target gray value of the central first pixel combination area to obtain 12 gray comparison results. Through binarization conversion, the first local texture value corresponding to the target local region is obtained.

[0119] In addition, the target local area can be divided according to the target local texture window of other specifications and sizes. The implementation process can be referred to the above examples, and will not be described in detail here.

[0120] In some implementations, to reduce the computational load and improve the efficiency of image texture value calculation when calculating the image texture value of video frames, the size parameters of the local texture window used to divide the local region of the video frame can be adjusted before calculating the image texture value, so as to obtain the target local texture window after adjustment. For example, step (103.2.1) may include: obtaining candidate local texture windows and determining the candidate window range value of the candidate local texture window; obtaining the target available resource amount of image computing power resources, and querying the window range list according to the target available resource amount to obtain the target window range value, wherein the window range list includes the correlation between different available resource amounts and window range values; when the candidate window range value is less than the target window range value, adjusting the candidate local texture window according to the target window range value to obtain the target local texture window; when the candidate window range value is greater than or equal to the target window range value, determining the candidate local texture window as the target local texture window. Here, the candidate local texture window can be a neighborhood region of a pre-set size range, which can be understood as the default operator coverage range of the local binarization mode algorithm. For example, the size of the candidate local texture window is a neighborhood range of 3 pixels by 3 pixels.

[0121] The image computing resources can be one or more of the computing resources used for image processing in the server, such as the central processing unit (CPU), graphics processing unit (GPU), memory, cache, etc. These computing resources affect the efficiency of calculating image texture values. The target available resource quantity can be the available quantity of any one or more of the above computing resources, for example, the available quantity of GPU computing resources.

[0122] The window range list can be a list of available resource quantities and window range values ​​for one or more image computing resources. Specifically, it includes the association between the available resource quantities and window range values ​​of one or more types of image computing resources. It should be noted that the server will monitor the remaining available resource quantity of local image computing resources in real time and synchronously update the available resource scalar of the corresponding type of image computing resources in the window range list according to the remaining available resource quantity.

[0123] Specifically, the default candidate local texture window corresponding to the local binarization mode algorithm is obtained, and the candidate window range value is determined, which represents the size of the candidate local texture window. Then, the target available resource amount of the image computing power resources on the server is obtained, and the window range list is queried according to the target available resource amount to determine the target window range value that matches the target available resource amount of the current image computing power resources. Finally, the target window range value is compared with the candidate window range value. If the target window range value is greater than the candidate window range value, the size of the candidate local texture window is adjusted according to the target window range value to obtain the adjusted target local texture window. Conversely, if the target window range value is less than or equal to the candidate window range value, the candidate local texture window is directly used as the target local texture window. Therefore, a texture window with a larger coverage area can be selected as the target local texture window. This will reduce the number of windows (i.e., "target local regions") when dividing the video frame into multiple target local regions according to the target local texture window. This will reduce the amount of computation required to calculate the image texture value, improve the efficiency of image texture value calculation, and effectively avoid delays in adjusting the encoding bitrate of the video encoder due to the image texture value calculation process, thus affecting the video encoding progress and efficiency. This method is reliable.

[0124] In this embodiment, to accelerate the calculation of image texture values ​​for video frames, in addition to increasing the size of the local texture window, the resolution of any "video frame" in this embodiment can also be reduced to decrease the computational load of image texture values, thereby improving the computational efficiency of image texture values ​​for video frames. It should be noted that reducing the image resolution of the video frame can reduce the computational load, and on this basis, further combining this with an enlarged target local texture window can further reduce the computational load of image texture values ​​for video frames, thereby further improving the computational efficiency of image texture values. This effectively avoids delays in subsequent video encoding efficiency due to the image texture value calculation process, ensuring reliability.

[0125] In some implementations, to reduce the computational load of the first video frame in calculating image texture values, the resolution of the first video frame can be reduced by downsampling, so that subsequent image texture value calculations can be performed based on the downsampled first video frame, thereby improving the computational efficiency of image texture values. For example, before step (103.1), the following steps may be included: obtaining the image resolution information corresponding to the first video frame currently to be encoded in the frame sequence to be encoded; performing downsampling processing on the first video frame based on the image resolution information to obtain the downsampled first video frame; then before step (103.1), the following steps may also be included: obtaining the grayscale value of each first pixel in the downsampled first video frame.

[0126] The image resolution information can be expressed as the amount of information contained in a video frame, represented by the number of pixels. Specifically, image resolution information can represent the number of pixels contained in a video frame per unit length (e.g., per inch), that is, the number of pixels contained in the video frame in both the length and width dimensions. It can be understood that image resolution information determines the level of detail in a video frame; the higher the resolution, the clearer the image.

[0127] It should be noted that pixel downsampling can reduce the computational load on the image texture value of the first video frame. Specifically, for the first video frame to be encoded in the current encoding progress of the frame sequence, the image resolution information of the first video frame can be obtained. This image resolution information represents the distribution and number of first pixels contained in the first video frame. Then, the pixel value corresponding to each first pixel in the first video frame is obtained. Based on the distribution of each first pixel in the first video frame as reflected by the image resolution information, downsampling is performed according to the size relationship of each first pixel value. For example, taking a "2-pixel by 2-pixel" or "3-pixel by 3-pixel" first pixel area as a unit, the first pixel with the largest pixel value in each first pixel area is used. In this way, the downsampled first video frame is obtained. Then, the grayscale value of each first pixel in the downsampled first video frame is obtained so that the first image texture value of the first video frame can be determined subsequently based on the grayscale value of each first pixel in the downsampled first video frame.

[0128] In this embodiment, due to limited network bandwidth resources, to reduce bandwidth consumption, if the difference between the first video frame and the second video frame in the frame sequence to be encoded is too small, the encoding bitrate used by the video encoder when encoding the previous video frame will continue to be used to encode the current video frame. The image quality of the currently encoded video frame will be similar to that of the previous video frame. Therefore, if the difference between the first video frame and the second video frame in the frame sequence to be encoded is too small, even if the texture complexity of the first video frame is greater than that of the second video frame, the encoding bitrate of the video encoder will not be adjusted. Instead, the encoding bitrate used by the video encoder when encoding the second video frame will be used to encode the first video frame. This reduces the frequency of bitrate adjustments during video encoding, thereby minimizing the bandwidth consumption of the encoded video frame stream data transmission.

[0129] In some implementations, to reduce the bandwidth consumption of the encoded video frame stream data, the similarity between the first video frame to be encoded and the second video frame that has already been encoded can be determined first. If the similarity is less than a certain threshold, the first image texture value of the first video frame is determined to determine the texture complexity of the first video frame. For example, before step 103, the following may also be included:

[0130] (103.a) Determine the similarity between the first video frame and the second video frame;

[0131] Then step 103 may include: when the similarity is less than a preset similarity threshold, determining the first image texture value of the first video frame currently to be encoded in the frame sequence to be encoded.

[0132] The preset similarity threshold can be used to determine the similarity between a first video frame to be encoded and a previously encoded second video frame, thereby determining the degree of difference between the first and second video frames. A higher similarity threshold indicates a smaller difference between the first and second video frames, and vice versa. This preset similarity threshold is used to determine the difference between the first and second video frames, deciding whether to determine the image texture values ​​of both frames, and thus further adjusting the encoding bitrate of the video encoder.

[0133] In determining the similarity between the first and second video frames, algorithms such as Hue Saturation Value (HSV), histogram statistics, and hash similarity statistics can be used. Any of these algorithms can be employed to determine the similarity between the first video frame (currently being encoded) and the second video frame (already encoded) in the sequence of frames to be encoded. Furthermore, after determining the similarity between the first and second video frames, this similarity is compared with a preset similarity threshold.

[0134] On the one hand, when the similarity between the first video frame and the second video frame is less than the preset similarity threshold, it indicates that the difference between the first video frame and the second video frame is too large. At this time, the first image texture value of the first video frame currently to be encoded in the frame sequence to be encoded is determined, and the second image texture value of the second video frame adjacent to the first video frame is determined, so that the encoding bitrate of the video encoder can be adjusted when the first image texture value is greater than the second image texture value.

[0135] On the other hand, when the similarity is greater than or equal to the preset similarity threshold, the video encoder encodes the first video frame currently to be encoded in the frame sequence to be encoded according to the current encoding bitrate (i.e. the encoding bitrate when encoding the second video frame).

[0136] In some implementations, a histogram statistical algorithm is used to calculate the color similarity between the first video frame and the second video frame. For example, the similarity includes color similarity, and step (103.a) may include:

[0137] (103.a.1) Obtain the first pixel value of each first pixel in the first video frame currently to be encoded in the frame sequence to be encoded, and generate a first histogram for the first video frame based on the statistical number of different first pixel values;

[0138] (103.a.2) Obtain the second pixel value of each second pixel point in the second video frame, and generate a second histogram for the second video frame based on the statistical number of different second pixel values;

[0139] (103.a.3) Determine the color similarity between the first video frame and the second video frame based on the difference between the first and second histograms.

[0140] Wherein, the first pixel value refers to the pixel value of the corresponding first pixel point in the first video frame, and the second pixel point refers to the pixel value of the corresponding second pixel point in the second video frame.

[0141] The first histogram is used to count the number of first pixels corresponding to each pixel value dimension of the first video frame. The width of the first histogram can include 256 pixel value dimensions, that is, pixel values ​​from 0 to 255. Each bar in the first histogram corresponds to a target pixel value, and the height of the bar represents the count of first pixels corresponding to that target pixel value in the first video. This first histogram can reflect the color characteristics of the first video frame. Similarly, the second histogram is used to count the number of second pixels corresponding to each pixel value dimension of the second video frame. The width of the second histogram can include 256 pixel value dimensions, that is, pixel values ​​from 0 to 255. This second histogram can reflect the color characteristics of the second video frame.

[0142] Color similarity can represent the degree of similarity in color between the first video frame and the second video frame, and also reflects the color difference between the first video frame and the second video frame.

[0143] Specifically, when determining the color similarity between the first and second video frames using the histogram statistical algorithm, firstly, the first pixel value of each first pixel in the first video frame is obtained, and the number of first pixels corresponding to each first pixel value dimension is counted. Each first pixel value dimension is used as the width (i.e., the horizontal axis value) of the histogram, and the number of first pixels corresponding to each first pixel value dimension is used as the height of the corresponding bar in the histogram, generating a first histogram for the first video frame to represent its color characteristics. Simultaneously, a second histogram corresponding to the second video frame is obtained. The generation process of this second histogram can refer to that of the first histogram; if it has already been generated in a historical timeframe, the second histogram corresponding to the second video frame can be directly obtained. Further, based on the differences between the first and second histograms in each pixel value dimension, the color similarity between the first and second video frames is calculated. In this way, color similarity is used to evaluate the degree of difference in color features between the first video frame and the second video frame, so as to make subsequent decisions on whether to adjust the encoding bitrate of the video encoder based on the texture difference between the first video frame and the second video frame.

[0144] In some implementations, the color similarity between a first video frame and a second video frame is determined based on the difference in the number of pixels in each pixel value dimension between a first histogram and a second histogram. For example, step (103.a.3) may include: determining a first statistic of the first video frame in each pixel value dimension based on the first histogram, and determining a second statistic of the second video frame in each pixel value dimension based on the second histogram; determining the difference in the number of pixels in each pixel value dimension between the first video frame and the second video frame based on the first and second statistics in each pixel value dimension; and determining the color similarity between the first video frame and the second video frame based on the difference in the number of pixels in each pixel value dimension.

[0145] The first statistic can be the number of first pixels corresponding to the first pixel value dimension in the first histogram, while the second statistic is the number of second pixels corresponding to the second pixel value dimension in the second histogram.

[0146] The pixel count difference refers to the difference in the number of pixels in the same pixel value dimension between the first histogram and the second histogram. For example, if the first histogram has 60 pixels with a value of 100 and the second histogram has 70 pixels with a value of 100, then the pixel count difference in the dimension with a pixel value of 100 is 30.

[0147] When determining the color similarity between the first and second video frames based on the difference in the number of pixels in each pixel value dimension, the process is as follows: First, the maximum value between the first and second statistics in that pixel value dimension is obtained as the target value. Then, for each pixel value dimension, a target ratio between the difference in the number of pixels and the target value is determined. This target ratio represents the proportion of difference between the first and second video frames in the current pixel value dimension, i.e., the degree of difference. The highest similarity is 1. By subtracting the target ratio from 1, the color sub-similarity between the first and second video frames in the current pixel value dimension is obtained. Finally, the color sub-similarity values ​​in each pixel value dimension are summed and averaged to obtain the overall color similarity.

[0148] It should be noted that the calculation process for the second image texture value of the second video frame can be found in the description of the "first video frame" above, and will not be repeated here.

[0149] By using the above methods, the complexity of the video frame corresponding to the current encoding progress and the complexity of the previous video frame can be determined during the video frame encoding process. This allows for the adjustment of the video encoder's bitrate according to frame-level decisions when it is determined that the texture complexity of the current video frame increases relative to the previous video frame, thus meeting the image quality requirements of the current video frame and ensuring reliability.

[0150] 104. When the texture complexity identified by the first image texture value is greater than the texture complexity identified by the second image texture value, the image texture ratio is determined based on the first image texture value and the second image texture value.

[0151] In this embodiment, after obtaining the first image texture value of the first video frame and the second image texture value of the second video frame, the texture complexity indicated by the first and second image texture values ​​is compared. This comparison of texture complexity between the first and second video frames helps determine whether to adjust the encoding bitrate of the video encoder. Given that a larger image texture value corresponds to greater texture complexity of the video frame, on the one hand, when the first image texture value is greater than the second image texture value, the encoding bitrate of the video encoder is adjusted; on the other hand, when the first image texture value is less than or equal to the second image texture value, the encoding bitrate of the video encoder is not adjusted. This ensures that the encoding bitrate of the video encoder can meet the image quality requirements of the encoded video frame for the first video frame being encoded, thus ensuring reliability.

[0152] The image texture ratio can be a ratio representing the texture complexity of the first video frame relative to the second video frame. It can be understood that when the texture complexity of the first video frame is greater than that of the second video frame, the image texture ratio is greater than 1, indicating that the first video frame is more complex than the second video frame.

[0153] It should be noted that when deciding on subsequent adjustments to the video encoder's bitrate, to ensure a reasonable adjustment, the ratio between the first and second image texture values ​​can be determined as the image texture ratio between the first and second video frames. This ratio allows for subsequent adjustments to the video encoder's bitrate, specifically adjusting the bitrate used when encoding the second video frame. This ensures that the adjusted bitrate meets the image quality requirements of the encoded first video frame, while reducing the bandwidth consumption of the video frame stream and improving reliability.

[0154] In some implementations, to avoid frequent adjustments to the video encoder's bitrate, when the first image texture value is greater than the second image texture value, an image texture ratio can be calculated based on the significant difference in texture complexity between the first and second video frames. This ratio can then be used to adjust the video encoder's bitrate subsequently. For example, step 104, "determining the image texture ratio based on the first and second image texture values," can include: obtaining the image texture difference between the first and second image texture values; and calculating the image texture ratio between the first and second image texture values ​​when the image texture difference is greater than a preset texture difference threshold.

[0155] The image texture difference can represent the degree of texture difference between the first video frame and the second video frame. Specifically, the texture difference is determined based on the texture values ​​of the first and second images. If the image texture difference is larger, it means that the texture difference between the first and second video frames is larger. Conversely, if the image texture difference is smaller, it means that the texture difference between the first and second video frames is smaller.

[0156] Specifically, the first image texture value is compared with the second image texture value. When the first image texture value is greater than the second image texture value, it means that the image texture of the first video frame to be encoded is more complex than the image texture of the previously encoded second video frame. If the difference in image texture complexity between the first video frame and the second video frame is small, the encoding bitrate used by the video encoder when encoding the second video frame can be used to encode the first video frame. This has little impact on the image quality of the first video frame and can reduce the amount of transmission bandwidth resources consumed.

[0157] Therefore, to avoid frequently increasing the encoding bitrate of the video encoder, when the first image texture value is greater than the second image texture value, the image texture difference between the first and second image texture values ​​can be determined and compared with a preset texture difference threshold. This preset texture difference threshold is used to determine the degree of difference between the first video frame to be encoded and the previously encoded second video frame. On the one hand, if the image texture difference is greater than the preset texture difference threshold, it indicates that the difference between the first and second video frames is large, and the previous encoding bitrate of the video frame encoder cannot meet the image quality requirements of the first video frame. In this case, it is necessary to determine the image texture ratio between the first and second image texture values ​​so that the previous encoding bitrate of the video encoder can be increased according to the image texture ratio. On the other hand, if the image texture difference is less than or equal to the preset texture difference threshold, it indicates that the difference between the first and second video frames is small, and the previous encoding bitrate of the video encoder does not need to be increased. The encoding bitrate used when encoding the second video frame can continue to be used for encoding the first video frame. This reduces the frequency of adjusting the encoding bitrate of the video encoder and also reduces the amount of transmission bandwidth resources required.

[0158] By comparing the first image texture value with the second image texture value, the comparison result of texture complexity between the first video frame and the second video frame can be determined. Thus, when the texture of the first video frame is more complex than that of the second video frame, it indicates that the texture complexity of the current video frame is increasing relative to the previous video frame. At this time, the decision is made to adjust the encoding bitrate of the video encoder according to the image texture ratio between the first video frame and the second video frame, so as to realize the adjustment of the encoding bitrate of the video encoder according to the frame level, and achieve more fine-grained and precise adjustment of the encoding bitrate of the video encoder.

[0159] 105. Adjust the encoding bitrate of the video encoder according to the image texture ratio to obtain the target encoding bitrate, and encode the first video frame according to the target encoding bitrate.

[0160] In this embodiment, after obtaining the image texture ratio between the first video frame and the second video frame, the previous encoding bitrate of the video encoder can be adjusted according to the image texture ratio. Since the encoding bitrate of the video encoder may be adjusted in real time during the encoding of the preceding video frames in the frame sequence to be encoded, the current encoding bitrate of the video encoder can be understood as the encoding bitrate used when encoding the second video frame. Therefore, adjusting the previous encoding bitrate of the video encoder according to the image texture ratio makes the adjustment range of the video encoder's encoding bitrate match the ratio of the texture complexity difference between the preceding and following video frames. This allows for a more refined and precise adjustment of the video encoder's encoding bitrate at the frame level, so that the current first video frame can be encoded according to the newly adjusted bitrate. Thus, while ensuring the image quality of the encoded first video frame, it is possible to maximize bandwidth resource savings and achieve reliability.

[0161] The target encoding bitrate can be the encoding bitrate that has been re-determined after the video encoder has been adjusted. This target encoding bitrate can be obtained by adjusting the previous encoding bitrate according to the image texture ratio. It is used to encode the first video frame in order to cope with the encoding of the first video frame with rich image content and complex information, and to ensure the image quality of the first video frame after encoding.

[0162] It should be noted that due to limited network bandwidth resources, it is necessary to make reasonable use of network bandwidth resources for the subsequent transmission of encoded video frames. This is to ensure the quality of video frames while avoiding playback stuttering and conserving network bandwidth resources as much as possible. Therefore, during the adjustment / bitrate control process of the video encoder, an upper limit can be set for the encoding bitrate to limit the video encoder's bitrate from exceeding this limit. This allows for the rational use of network bandwidth resources, ensuring the quality of video frames while avoiding playback stuttering.

[0163] In some implementations, the bitrate of the video encoder is adjusted based on the image-texture ratio, and the adjusted bitrate is compared with the limited bitrate of the video encoder to determine the final target bitrate of the video encoder. For example, step 105, "adjusting the bitrate of the video encoder based on the image-texture ratio to obtain the target bitrate," may include: obtaining an upper limit value for the bitrate of the video encoder; weighting the bitrate based on the image-texture ratio to obtain candidate bitrates; when the upper limit value is greater than the candidate bitrates, adjusting the bitrate of the video encoder according to the candidate bitrates to obtain the target bitrate; when the upper limit value is less than the candidate bitrates, adjusting the bitrate of the video encoder according to the upper limit value to obtain the target bitrate.

[0164] The upper limit of the encoding bitrate can be the bitrate limit of the video encoder, which can be understood as the maximum encoding bitrate of the video encoder. This upper limit of the encoding bitrate can be set according to the available bandwidth resources or the expected target bandwidth resources, so that the maximum encoding bitrate cannot exceed the upper limit of the encoding bitrate during the bitrate adjustment (i.e., bitrate control) process. In this way, the video frame quality is maximized, and the encoded video frame stream is transmitted smoothly under the limited bandwidth resources, avoiding the phenomenon of video playback stuttering, and has reliability.

[0165] Specifically, when determining to adjust the encoding bitrate of the video encoder, the current encoding bitrate of the video encoder is obtained, for example, the encoding bitrate used when encoding the second video frame, and the upper limit of the encoding bitrate for the video encoder is also obtained. Further, the encoding bitrate is weighted according to the image texture ratio, and the adjustment range is based on the proportion of texture complexity differences between consecutive video frames, increasing the encoding bitrate of the video encoder. This achieves frame-level adjustment of the video encoder's encoding bitrate, obtaining candidate encoding bitrates. Further, the upper limit of the video encoder's encoding bitrate is used as a bitrate adjustment limit threshold. If the adjusted candidate encoding bitrate is greater than or equal to the upper limit of the video encoder's encoding bitrate, then the upper limit of the video encoder's encoding bitrate is used as the target encoding bitrate. Conversely, if the adjusted candidate encoding bitrate is less than the upper limit of the video encoder's encoding bitrate, then the video encoder's encoding bitrate is adjusted according to the candidate encoding bitrate, i.e., the candidate encoding bitrate is used as the target encoding bitrate for the video encoder.

[0166] In some implementations, the video encoding process encodes video frames sequentially in the sequence of frames to be encoded. Other video frames following the first video frame in the sequence also need to be encoded according to the encoding process of the first video frame. For example, after "encoding the first video frame according to the target encoding bitrate" in step 105, the process may further include: obtaining the third image texture value of a third video frame located after and adjacent to the first video frame in the sequence of frames to be encoded; when the third image texture value is detected to be greater than the first image texture value, determining the target image texture ratio between the third video frame and the first video frame; adjusting the target encoding bitrate of the video encoder according to the target image texture ratio to obtain the adjusted target encoding bitrate, and encoding the third video frame according to the adjusted target encoding bitrate; when the third image texture value is detected to be less than or equal to the first image texture value, encoding the third video frame according to the target encoding bitrate.

[0167] The third video frame can be a video frame in the frame sequence to be encoded that is temporally following the first video frame, i.e., the video frame after the first video frame. The third image texture value can represent the image texture complexity of the third video frame. Assuming there are N temporally arranged video frames in the frame sequence to be encoded, the first video frame can be the i-th video frame, the second video frame can be the (i+1)-th video frame, where i is greater than 1 and less than N.

[0168] Specifically, after encoding the first video frame in the sequence of frames to be encoded, the next step is to encode the next frame (i.e., the third video frame). Before encoding the third video frame, the third image texture value of the third video frame can be determined and compared with the first image texture value.

[0169] On one hand, if the texture value of the third image is greater than that of the first image, it indicates that the texture of the third video frame is more complex and the content of the video frame is richer than that of the first video frame. In this case, the decision is made to adjust the encoding bitrate of the video encoder. At this time, the ratio between the texture value of the third image and the texture value of the first image is calculated as the target image texture ratio between the third video frame and the first video frame. The target encoding bitrate of the video encoder is adjusted according to the target image texture ratio. This target encoding bitrate is the bitrate of the video encoder when encoding the first video frame, and a target candidate encoding bitrate is obtained. At this time, the target candidate encoding bitrate can also be compared with the upper limit of the encoding bitrate. If the target candidate encoding bitrate is greater than the upper limit of the encoding bitrate, the upper limit of the encoding bitrate is determined as the bitrate for encoding the third video frame, and the adjustment of the current encoding bitrate of the video encoder is rejected. If the target candidate encoding bitrate is less than the upper limit of the encoding bitrate, the target candidate encoding bitrate is used as the adjusted target encoding bitrate of the video encoder. Further, the third video frame is encoded by the video encoder according to the adjusted target encoding bitrate.

[0170] On the other hand, if the texture value of the third image is less than or equal to the texture value of the first image, the target encoding bitrate previously used by the video encoder is used to encode the third video frame to ensure the quality of the video image after encoding, thus ensuring reliability.

[0171] By using the above method, the encoding bitrate of the video encoder can be adjusted according to the image texture ratio between consecutive video frames. This ensures that the adjustment range of the video encoder's encoding bitrate matches the ratio of the texture complexity differences between consecutive video frames. This allows for more delicate and precise adjustment of the video encoder's encoding bitrate at the frame level. The current first video frame is then encoded according to the newly adjusted bitrate, maximizing bandwidth savings while ensuring the image quality of the encoded first video frame, thus providing reliability.

[0172] As can be seen from the overall description of the embodiments of this application, the embodiments of this application can obtain the current encoding bitrate of the video encoder; obtain the sequence of frames to be encoded of the target video source, and start encoding the first video frame in the sequence of frames to be encoded according to the encoding bitrate by the video encoder; determine the first image texture value of the first video frame currently to be encoded in the sequence of frames to be encoded, and determine the second image texture value of the second video frame adjacent to the first video frame, wherein the first video frame is any video frame after the first video frame; when the first image texture value is greater than the second image texture value, determine the image texture ratio according to the first image texture value and the second image texture value; adjust the encoding bitrate of the video encoder according to the image texture ratio to obtain the target encoding bitrate, and encode the first video frame according to the target encoding bitrate.

[0173] Based on this, this application can obtain the current encoding bitrate of the video encoder in real time, and then encode the video frames in the current frame sequence of the target video source according to the encoding bitrate. Furthermore, during the encoding process, the texture complexity between the first video frame to be encoded and the adjacent second video frame is compared in real time. Texture complexity represents the richness of the video frame's content to evaluate the required encoding bitrate. If the texture of the first video frame is more complex than the previous frame, the image texture ratio between the first and second video frames is determined based on the first and second image texture values. Finally, the encoding bitrate of the video encoder is adjusted to the target bitrate according to the image texture ratio. The target encoding bitrate is set so that the video encoder encodes the first video frame according to the adjusted target encoding bitrate. In this way, compared with the related technology where the video encoder encodes video frames according to the video source bitrate counted over a fixed duration, this application can compare the texture complexity between the preceding and following video frames during the encoding process of the target video source's frame sequence. When the texture of the following video frame is more complex than that of the preceding video frame, the encoding bitrate of the video encoder is adjusted according to the texture ratio between the preceding and following video frames. In this way, the encoding bitrate of the video encoder can be flexibly and finely adjusted at the frame level to ensure the encoding quality of the following video frames and guarantee the video frame image quality of the video encoding.

[0174] Based on the methods described in the above embodiments, the following examples will provide further detailed explanations.

[0175] Figure 4 is a schematic flowchart of another step of the video encoding method provided in this application embodiment. For ease of understanding, this application embodiment is described in conjunction with Figure 4.

[0176] In this embodiment, the description will focus on a video encoding device, which can be integrated into a computer device such as a terminal or server. For example, when the processor on the computer device executes the program corresponding to the video encoding method, the specific flow of the video encoding method is as follows:

[0177] 201. Obtain the current video source bitrate of the target video source, and update the encoding bitrate of the video encoder synchronously according to the video source bitrate.

[0178] In this embodiment, the encoding bitrate of the video encoder is adjustable. To ensure a reasonable encoding bitrate, the bitrate can be adjusted based on the source video bitrate, resulting in better image quality for more complex video frames after encoding. Specifically, this can be achieved by reducing the granularity of the source video bitrate acquisition to acquire the source video bitrate of the target video source and simultaneously adjusting the encoding bitrate of the video encoder. This allows for finer adjustments to the encoding bitrate of the video encoder, serving as a reliable base bitrate for video encoding.

[0179] Specifically, the bitrate of the target video source can be determined based on the interval between two adjacent key video frames, thereby adjusting the encoding bitrate of the video encoder. It should be noted that the video frame stream data of the target video source can be organized and transmitted to the server in the form of a video frame sequence. This sequence can include different types of video frames, such as key video frames (Intra-coded Frames, I-frames), predicted frames (P-frames), and bi-directional predicted frames (B-frames). The bitrate between two key frames can be statistically determined based on the interval between them (group of pictures, GOP).

[0180] Specifically, the frame rate of the target video source is determined, and the interval between two adjacent key video frames in the transmission sequence of the target video source is detected in real time, such as the interval length or interval duration. Then, the video source bitrate is calculated based on the interval between the two adjacent key video frames and the frame rate, for example, by determining the video source bitrate based on the ratio between the frame rate and the interval. Further, the encoding bitrate in the video encoder is synchronously updated according to the video source bitrate, so that the current encoding bitrate can adapt to the sequence of frames to be encoded between the current two key frames in the video frame sequence, which can be understood as a sub-sequence or segment sequence. Based on this, since the real-time video frame stream data of the target video source is organized in sequence form, the video source bitrate in the real-time stage is determined by using the interval between key frames as the base bitrate for encoding by the video encoder, which better matches the encoding bitrate requirements of each video frame sequence, providing flexibility and reliability.

[0181] 202. Obtain the sequence of frames to be encoded from the target video source, and start encoding the first video frame in the sequence of frames to be encoded using a video encoder according to the encoding bitrate.

[0182] In this embodiment, after obtaining the real-time encoding bitrate of the video encoder, the sequence of frames to be encoded corresponding to the target video source can be obtained. The video encoder can then start encoding the video frames in the sequence of frames to be encoded according to the current encoding bitrate. Since the encoding process of the video encoder can be frame-by-frame, the first video frame in the sequence of frames to be encoded can be encoded according to the encoding bitrate to compress the video frames, increase the data transmission rate per unit time under a specific bandwidth, and ensure the smoothness of video playback at the playback end.

[0183] It should be noted that when the server performs video encoding, it can simultaneously receive and acquire video frame stream data from the target video source and encode the acquired video frame stream data in real time. Assuming the video frame stream data is transmitted in sequence, the server can sequentially receive and acquire each sequence of frames to be encoded from the target video source and encode the video frames in each sequence sequentially. Therefore, the encoding process of the target video source's video stream data is performed in real time. If the server has already encoded the video frames in the historical sequence of frames to be encoded, when the server acquires a sequence of frames to be encoded, and the video encoder in the server has been updated and adjusted to the corresponding encoding bitrate, the video encoder uses the current encoding bitrate as the base bitrate, that is, it starts encoding the first video frame in the sequence of frames to be encoded according to the current real-time encoding bitrate.

[0184] 203. Determine the similarity between the first video frame currently to be encoded in the frame sequence and the second video frame that is adjacent to the first video frame.

[0185] In this embodiment, due to limited network bandwidth resources, to reduce bandwidth consumption, if the difference between the first video frame and the second video frame in the frame sequence to be encoded is too small, the encoding bitrate used by the video encoder when encoding the previous video frame will continue to be used to encode the current video frame. The image quality of the currently encoded video frame will be similar to that of the previous video frame. Therefore, if the difference between the first video frame and the second video frame in the frame sequence to be encoded is too small, even if the texture complexity of the first video frame is greater than that of the second video frame, the encoding bitrate of the video encoder will not be adjusted. Instead, the encoding bitrate used by the video encoder when encoding the second video frame will be used to encode the first video frame. This reduces the frequency of bitrate adjustments during video encoding, thereby minimizing the bandwidth consumption of the encoded video frame stream data transmission.

[0186] The similarity can refer to the color similarity between the first video frame and the second video frame.

[0187] Specifically, when determining the color similarity between the first and second video frames using the histogram statistical algorithm, firstly, the first pixel value of each first pixel in the first video frame is obtained, and the number of first pixels corresponding to each first pixel value dimension is counted. Each first pixel value dimension is used as the width (i.e., the horizontal axis value) of the histogram, and the number of first pixels corresponding to each first pixel value dimension is used as the height of the corresponding bar in the histogram, generating a first histogram for the first video frame to represent its color characteristics. Simultaneously, a second histogram corresponding to the second video frame is obtained. The generation process of this second histogram can refer to that of the first histogram; if it has already been generated in a historical time period, the second histogram corresponding to the second video frame can be directly obtained. Furthermore, the first statistical count of pixels in each pixel value dimension of the first histogram is determined, and the second statistical count of pixels in each pixel value dimension of the second histogram is determined. Based on the first and second statistical counts in each pixel value dimension, the difference in the number of pixels between the first video frame and the second video frame in each pixel value dimension is determined. Based on the difference in the number of pixels in each pixel value dimension, the color similarity between the first video frame and the second video frame is determined.

[0188] In this way, color similarity is used to evaluate the degree of difference in color features between the first video frame and the second video frame, so as to make subsequent decisions on whether to adjust the encoding bitrate of the video encoder based on the texture difference between the first video frame and the second video frame.

[0189] 204. When the similarity is less than the preset similarity threshold, determine the first image texture value of the first video frame and the second image texture value of the second video frame.

[0190] In this embodiment, the similarity between the first video frame and the second video frame is compared with a preset similarity threshold. This preset similarity threshold is used to determine the degree of similarity between the first video frame to be encoded and the previously encoded second video frame, thereby determining the degree of difference between the first and second video frames. A higher similarity indicates a smaller difference between the first and second video frames, and vice versa. The difference between the first and second video frames is determined using this preset similarity threshold to decide whether it is necessary to determine the image texture values ​​of the first and second video frames.

[0191] On the one hand, when the similarity between the first video frame and the second video frame is less than the preset similarity threshold, it indicates that the difference between the first video frame and the second video frame is too large. At this time, the first image texture value of the first video frame currently to be encoded in the frame sequence to be encoded is determined, and the second image texture value of the second video frame adjacent to the first video frame is determined, so that the encoding bitrate of the video encoder can be adjusted when the first image texture value is greater than the second image texture value.

[0192] On the other hand, when the similarity is greater than or equal to the preset similarity threshold, the video encoder encodes the first video frame currently to be encoded in the frame sequence to be encoded according to the current encoding bitrate (i.e. the encoding bitrate when encoding the second video frame).

[0193] 205. When the texture value of the first image is greater than the texture value of the second image, determine the image texture ratio based on the texture value of the first image and the texture value of the second image.

[0194] In this embodiment of the application, after obtaining the first image texture value of the first video frame and the second image texture value of the second video frame, the first image texture value and the second image texture value are compared to determine the texture complexity between the first video frame and the second video frame. This step takes the example that the larger the image texture value, the greater the texture complexity.

[0195] If the image texture difference is greater than a preset texture difference threshold, it indicates a significant difference between the first and second video frames. The previous encoding bitrate of the video frame encoder is insufficient to meet the image quality requirements of the first video frame. In this case, it is necessary to determine the image texture ratio between the first and second image texture values ​​so that the previous encoding bitrate of the video encoder can be increased accordingly. Conversely, if the image texture difference is less than or equal to the preset texture difference threshold, it indicates a smaller difference between the first and second video frames. In this case, the previous encoding bitrate of the video encoder does not need to be increased, and the encoding bitrate used for encoding the second video frame can continue to be used for encoding the first video frame. This reduces the frequency of bitrate adjustments by the video encoder and decreases the consumption of transmission bandwidth resources.

[0196] 206. Adjust the encoding bitrate of the video encoder according to the image texture ratio to obtain the target encoding bitrate, and encode the first video frame according to the target encoding bitrate.

[0197] In this embodiment, the previous encoding bitrate of the video encoder can be adjusted according to the image texture ratio. Since the encoding bitrate of the video encoder may be adjusted in real time during the encoding of the preceding video frames in the frame sequence to be encoded, the current encoding bitrate of the video encoder can be understood as the encoding bitrate used when encoding the second video frame. Therefore, adjusting the previous encoding bitrate of the video encoder according to the image texture ratio makes the adjustment range of the video encoder's encoding bitrate match the ratio of the texture complexity difference between the preceding and following video frames. This allows for a more delicate and precise adjustment of the video encoder's encoding bitrate at the frame level, so that the current first video frame can be encoded according to the newly adjusted bitrate. Thus, while ensuring the image quality of the encoded first video frame, it is possible to maximize the saving of bandwidth resources and achieve reliability.

[0198] Specifically, when determining to adjust the encoding bitrate of the video encoder, the current encoding bitrate of the video encoder is obtained, for example, the encoding bitrate used when encoding the second video frame, and the upper limit of the encoding bitrate for the video encoder is also obtained. Further, the encoding bitrate is weighted according to the image texture ratio, and the adjustment range is based on the proportion of texture complexity differences between consecutive video frames, increasing the encoding bitrate of the video encoder. This achieves frame-level adjustment of the video encoder's encoding bitrate, obtaining candidate encoding bitrates. Further, the upper limit of the video encoder's encoding bitrate is used as a bitrate adjustment limit threshold. If the adjusted candidate encoding bitrate is greater than or equal to the upper limit of the video encoder's encoding bitrate, then the upper limit of the video encoder's encoding bitrate is used as the target encoding bitrate. Conversely, if the adjusted candidate encoding bitrate is less than the upper limit of the video encoder's encoding bitrate, then the video encoder's encoding bitrate is adjusted according to the candidate encoding bitrate, i.e., the candidate encoding bitrate is used as the target encoding bitrate for the video encoder.

[0199] The upper limit of the encoding bitrate can be the bitrate limit of the video encoder, which can be understood as the maximum encoding bitrate of the video encoder. This upper limit of the encoding bitrate can be set according to the available bandwidth resources or the expected target bandwidth resources, so that the maximum encoding bitrate cannot exceed the upper limit of the encoding bitrate during the bitrate adjustment (i.e., bitrate control) process. In this way, the video frame quality is maximized, and the encoded video frame stream is transmitted smoothly under the limited bandwidth resources, avoiding the phenomenon of video playback stuttering, and has reliability.

[0200] By using the above method, the encoding bitrate of the video encoder can be adjusted according to the image texture ratio between consecutive video frames. This ensures that the adjustment range of the video encoder's encoding bitrate matches the ratio of the texture complexity differences between consecutive video frames. This allows for more delicate and precise adjustment of the video encoder's encoding bitrate at the frame level. The current first video frame is then encoded according to the newly adjusted bitrate, maximizing bandwidth savings while ensuring the image quality of the encoded first video frame, thus providing reliability.

[0201] To facilitate understanding of the embodiments of this application, specific application scenario examples will be used to describe the embodiments of this application. Specifically, the application scenario example will be described by performing the above steps 201-206.

[0202] It should be noted that this video encoding method is suitable for scenarios such as live video streaming and real-time audio and video, including game streaming, game footage, multiplayer cooperative gameplay, and online movie viewing. This video encoding primarily compresses video frames in a live stream to increase the data throughput per unit time, which can be understood as the data transmission volume. The goal is to achieve an optimal balance between video quality and the reduction of buffering issues for users while meeting target bandwidth requirements, thereby improving video quality. The following section introduces this video encoding method using an example scenario:

[0203] I. A brief introduction to the video encoding scenario example is as follows:

[0204] Video encoding can be implemented based on video codec control (i.e., bitrate control). Video codec control analyzes the complexity of the video and the number of remaining bits to adjust the quantization parameters of the image layer and coding unit layer during encoding. This allows the encoder to achieve an optimal balance between video quality and the duration of buffering for the user, while meeting the target bandwidth (e.g., a range similar to the video source bitrate). Therefore, in live streaming or real-time audio / video scenarios, the encoder's bitrate can be adjusted based on the source video bitrate to ensure video quality.

[0205] This video encoding scenario example is based on frame-level bitrate calculation to adjust and control the encoder's encoding bitrate. This allows the encoder to achieve the optimal balance between video quality and the duration of buffering for the user while meeting the target bandwidth (e.g., a range of values ​​similar to the video source bitrate), thus ensuring video quality.

[0206] Figure 5 is an example diagram of a video encoding scenario provided in an embodiment of this application. To optimize the allocation of video data transmission within limited bandwidth resources, as shown in Figure 5, the specific video encoding scenario is as follows: A video source is input, and the video source is transmitted as a sequence of video frames (images). The sequence includes multiple video frames. During the encoding process of each video frame, pre-analysis can be performed to determine the spatiotemporal complexity of the video frame. Based on the spatiotemporal complexity of the video frame, the encoder's bitrate is controlled. Specifically, the quantization parameter (QP) of the image layer in the encoder and the quantization parameter (QP) of the encoding unit in the encoder are adjusted. This compresses and buffers the actual bitrate of the encoder, allowing the video frames to be encoded at the target bitrate. Thus, the video quality is optimized as much as possible within the target bandwidth.

[0207] Figure 6 is an example diagram of bitrate variation in a bitrate control scenario for video encoding provided in this application embodiment. Referring to Figure 6, during the video encoding process, bitrate control is not performed between stages A and B. During the stage from A to B, the actual bitrate fluctuates, sometimes high and sometimes low, and is unstable. The actual bitrate may exceed the target bandwidth limit. To ensure the encoded bitrate meets the target bandwidth requirement, bitrate control can be performed. After bitrate control, the output bitrate is generally below the target bandwidth limit, and the output bitrate tends to be stable without significant fluctuations, effectively ensuring the quality of the encoded video and providing reliability.

[0208] II. The specific implementation process of this video encoding scenario example is as follows:

[0209] This video encoding scenario example is not limited to live video streaming and video-on-demand scenarios. It supports encoding video media stream data in various data formats, such as Real-Time Messaging Protocol (RTMP), Flash Video (FLV), HTTP Live Streaming (HLS), Dynamic Adaptive Streaming over HTTP (DASH), Web Real-Time Communications (WebRTC), Transport Stream (TS), and Moving Picture Experts Group 4 (MP4). Taking live video streaming as an example, the broadcaster's client can transmit video stream data to the video encoding server in real time. The video encoding server then encodes the live video stream. The live stream can be an image stream composed of multiple consecutive video frames organized as a frame sequence. The video encoding server encodes each video frame in the frame sequence. This encoding process can be understood as compression, which helps to increase data throughput within limited bandwidth resources, ensuring smooth and uninterrupted live video streaming.

[0210] The video encoding process can be divided into two parts: bitrate control and adjustment and video frame encoding. This video encoding scenario example will focus on the bitrate control and adjustment process.

[0211] (1) Encoding rate control adjustment

[0212] Figure 7 is an example diagram of the video encoding process provided in an embodiment of this application. Referring to Figure 7, the bitrate control adjustment during the video encoding process is described in detail below:

[0213] Video frames are organized in sequences. The first video frame in a sequence is called an Instantaneous Decoder Refresh (IDR) video frame. IDR frames are all keyframe (I-frame) images. A sequence of frames to be encoded can contain many I-frame images. Images after I-frame images can use images between I-frame images as motion references.

[0214] (1.1) Calculate the video source bitrate based on the interval between keyframes.

[0215] In live video streaming or real-time audio / video scenarios, to meet the bitrate requirements of the video encoding process, the video encoder's bitrate can be initially adjusted based on the video source bitrate. This bitrate serves as the base bitrate for the current frame sequence to be encoded, allowing for further bitrate adjustments. The video source bitrate can be determined in real-time based on the frame interval (group of pictures, GOP) between two keyframes (I-frames). The frame interval represents a group of video frames (the group of frames to be encoded). Specifically, the frame length between the two keyframes (I-frames) and the frame rate of the playback client are first determined. Then, the number of video frame groups per unit time, i.e., the video source bitrate, is determined based on the ratio between the frame rate and the frame length. This bitrate is then dynamically loaded and updated into the video encoding kernel (i.e., the video encoder) to ensure the encoder obtains the current base bitrate.

[0216] (1.2) Detect any abrupt changes between any two adjacent video frames at the frame level. Detection methods can include Hue Saturation Value (HSV), histogram statistics, hash similarity statistics, etc. Taking the detection of abrupt changes between any two adjacent video frames using frame-level histograms as an example, the specific steps are as follows:

[0217] For two adjacent video frames, a histogram for each video frame is calculated and generated, as follows: the pixel value of each pixel in the video frame is calculated, and according to the scalar of 256 pixel values ​​from 0 to 255, the number of pixels corresponding to each pixel value is counted. Thus, a histogram of color for the video frame is generated based on the number of pixels corresponding to each pixel value.

[0218] Furthermore, based on the histograms of the two video frames, the color similarity between the video frames is calculated, as shown below:

[0219] Among them, S h This represents the color similarity between two video frames. A similarity score closer to 1 indicates greater similarity between the two frames, while a similarity score closer to 0 indicates greater difference. 'n' represents the histogram width, with a maximum width of 256. i Let b represent the value of pixel i in the histogram for one of the video frames (i.e., the statistical value of the pixel). i This represents the pixel value i of one of the video frames in the histogram.

[0220] Thus, by using the above method, the similarity between two video frames can be determined, thereby determining the difference between the two video frames. Specifically, this can be determined according to "1-S". h "" indicates the difference value.

[0221] (1.3) For two adjacent video frames, if the difference exceeds 20%, the texture complexity of these two video frames is calculated. For example, methods for analyzing image texture complexity include Euclidean distance, statistical histograms, Local Binary Patterns (LBP) detection algorithms, and Convolutional Neural Networks (CNN) feature extraction and classification algorithms. Taking the LBP detection algorithm as an example, Local Binary Patterns is an operator used to describe local features of an image. LBP features have significant advantages such as grayscale invariance and rotation invariance. The specific process is as follows:

[0222] Figure 8 is an example diagram of the image texture value calculation scenario provided in the embodiments of this application. Referring to Figure 8, the calculation process of the image texture value is described. The original LBP operator is defined as follows: within a 3*3 window, using the center pixel as a threshold, the grayscale values ​​of the eight adjacent pixels are compared with it. If the value of the surrounding pixels is greater than the value of the center pixel, the position of that pixel is marked as 1; otherwise, it is marked as 0. Thus, the comparison of the eight points within the 3*3 neighborhood generates an 8-bit binary number (usually converted to decimal, i.e., LBP code, of which there are 256 possible values), which is the LBP value of the center pixel of the window, as shown in Figure 8, resulting in (01111100). 10 =124, and this value is used to reflect the texture information of the area. The LBP values ​​of the areas detected in each frame are combined to calculate the texture LEB value of the video frame.

[0223] It should be noted that when detecting image texture values ​​in video frames, the smaller the operator window, the more windows a video frame can be divided into, and the greater the computational load. Conversely, if the window is larger, fewer windows are divided, and the computational load will be relatively reduced. Therefore, the size of the detection area of ​​the video frame can be automatically adjusted according to the video frame resolution and computing power to reduce the computational load when calculating image texture values.

[0224] It's important to note that when detecting image texture values ​​in video frames, the higher the video resolution, the more windows are needed for the same window size, resulting in a higher computational load. Therefore, the video resolution can be adjusted by downsampling. For example, downsampling a 4K video frame to obtain a 2K video frame reduces the number of windows by a factor of four, thus reducing the computational load by a factor of four. In this way, a 2K video frame can be used as a window to calculate texture values, replacing the window calculations from the 4K video frame, and obtaining a similar value.

[0225] (1.4) The higher the texture complexity, the higher the bitrate consumed by encoding. If the image texture value of the current video frame is greater than that of the previous video frame, the encoding bitrate of the video encoder is adjusted. For example, if the image texture value of the current video frame is LEB1 and the image texture value of the previous video frame is LEB0, and if LEB1 > LEB0, the video encoding bitrate of the video encoding kernel is updated and increased in real time. Let the encoding bitrate of the current video encoding kernel be bitrate, and the maximum configurable video bitrate of the video encoding kernel be max-bitrate. Then, the encoding bitrate of the video encoding kernel is adjusted to obtain the target encoding bitrate, specifically expressed as: bitrate = min(max-bitrate, bitrate * LEB1 / LEB0).

[0226] In this video encoding scenario, when encoding multiple video frames in the same sequence, the bitrate can be updated iteratively, and the encoding bitrate is adjusted based on the bitrate of the previous frame.

[0227] (2) Video frame encoding. Specifically, for each video frame in the sequence of frames to be encoded, each frame is encoded sequentially in a step-by-step manner. During the encoding process, the encoding bitrate of the video encoding kernel is adjusted according to the process (1.1) to (1.4) in (1) above, so as to encode the video frame of the current frame according to the adjusted encoding bitrate.

[0228] In the video encoding scenario of the entire live video or video-on-demand, the encoding bitrate control adjustment is performed according to (1) and (2) video frame encoding to complete the encoding of the entire live video or video-on-demand process until the end. Real-time dynamic detection is performed to detect whether the video screen suddenly switches from a static or relatively low bitrate scene to a scene with a higher bitrate and more complex picture. If so, the video encoding bitrate is adjusted in real time, only increasing and not decreasing, to solve the problem of statistical delay in the existing video bitrate statistics method caused by such scene switching. Due to insufficient video bitrate statistics, the video encoding kernel bitrate allocation is insufficient, resulting in poor picture quality.

[0229] By executing the above video encoding scenario example, the following effects can be achieved: Frame-level bitrate calculation optimizes and improves image quality bitrate control, ensuring that the video encoder's bitrate is as close as possible to the video source bitrate, while not exceeding the set target bandwidth. In this way, once the video scene stabilizes, fixed-duration bitrate statistics can guarantee that the statistical video bitrate will not exceed the original stream, preventing bitrate from getting out of control due to real-time frame dynamic adjustments and increasing too much, ultimately leading to an excessively high video bitrate that negatively impacts the viewing experience for subsequent viewers.

[0230] As described above, this embodiment of the application can obtain the current encoding bitrate of the video encoder in real time. Then, it encodes the video frames in the current frame sequence to be encoded of the target video source according to the encoding bitrate. Furthermore, during the encoding process, the texture complexity between the first video frame to be encoded and the previous adjacent second video frame is compared in real time. The texture complexity represents the richness of the video frame's image content to evaluate the required encoding bitrate. If the texture of the current first video frame is more complex than the previous frame, the image texture ratio between the first and second video frames is determined based on the first and second image texture values. Finally, the encoding bitrate of the video encoder is adjusted according to the image texture ratio. The target encoding bitrate is used to enable the video encoder to encode the first video frame according to the adjusted target encoding bitrate. In this way, compared with the related technology where the video encoder encodes video frames according to the video source bitrate counted over a fixed duration, this application can compare the texture complexity between the preceding and following video frames during the encoding process of the target video source's frame sequence. When the texture of the following video frame is more complex than that of the preceding video frame, the encoding bitrate of the video encoder is adjusted according to the texture ratio between the preceding and following video frames. In this way, the encoding bitrate of the video encoder can be flexibly and finely adjusted at the frame level to ensure the encoding quality of the following video frames by the video encoder and guarantee the video frame image quality of the video encoding.

[0231] For details on the implementation of each of the above steps, please refer to the previous examples, which will not be repeated here.

[0232] To facilitate better implementation of the video encoding method provided in this application, this application also provides an apparatus based on the above-described video encoding method. The meanings of the terms used are the same as in the video encoding method described above, and specific implementation details can be found in the descriptions within the method embodiments.

[0233] Please refer to Figure 9, which is a schematic diagram of the structure of a video encoding device provided in an embodiment of this application. The video encoding device is integrated into the computer equipment of this application. The video encoding device may include a first acquisition unit 401, an encoding unit 402, a first determination unit 403, a second determination unit 404, and an adjustment unit 405.

[0234] Acquisition unit 401 is used to acquire the current encoding bitrate of the video encoder;

[0235] The encoding unit 402 is used to acquire the sequence of frames to be encoded from the target video source and encode the first video frame in the sequence of frames to be encoded according to the encoding bitrate by the video encoder.

[0236] The first determining unit 403 is used to determine the first image texture value of the first video frame currently to be encoded in the frame sequence to be encoded, and to determine the second image texture value of the second video frame adjacent to the first video frame. The first video frame is any video frame after the first video frame.

[0237] The second determining unit 404 is used to determine the image texture ratio based on the first image texture value and the second image texture value when the texture complexity identified by the first image texture value is greater than the texture complexity identified by the second image texture value.

[0238] The adjustment unit 405 is used to adjust the encoding bitrate of the video encoder according to the image texture ratio to obtain the target encoding bitrate, and to encode the first video frame according to the target encoding bitrate.

[0239] In some embodiments, the video encoding apparatus further includes a third determining unit, used for:

[0240] Determine the similarity between the first video frame and the second video frame;

[0241] The first determining unit 403 is also used for:

[0242] When the similarity is less than a preset similarity threshold, the first image texture value of the first video frame to be encoded in the frame sequence is determined.

[0243] In some implementations, the similarity includes color similarity, and the third determining unit is further used for:

[0244] Obtain the first pixel value of each first pixel in the first video frame, and generate a first histogram for the first video frame based on the statistical count of different first pixel values.

[0245] Obtain the second pixel value of each second pixel point in the second video frame, and generate a second histogram for the second video frame based on the statistical count of different second pixel values;

[0246] The color similarity between the first and second video frames is determined based on the difference between the first and second histograms.

[0247] In some implementations, the third determining unit is further configured to:

[0248] The first statistic of the first video frame in each pixel value dimension is determined based on the first histogram, and the second statistic of the second video frame in each pixel value dimension is determined based on the second histogram.

[0249] Based on the first and second statistics in each pixel value dimension, determine the difference in the number of pixels between the first video frame and the second video frame in each pixel value dimension;

[0250] The color similarity between the first video frame and the second video frame is determined based on the difference in the number of pixels in each pixel value dimension.

[0251] In some embodiments, the second determining unit 404 is further configured to:

[0252] Obtain the image texture difference between the first image texture value and the second image texture value;

[0253] When the image texture difference is greater than the preset texture difference threshold, the image texture ratio of the first image texture value and the second image texture value is calculated.

[0254] In some embodiments, the first determining unit 403 is further configured to:

[0255] Obtain the grayscale value of each first pixel in the first video frame;

[0256] The first image texture value of the first video frame is determined based on the grayscale value of each first pixel.

[0257] In some embodiments, the first determining unit 403 is further configured to:

[0258] Get the target local texture window;

[0259] According to the target local texture window, the first pixel in the first video frame is divided to obtain multiple target local regions.

[0260] Based on the grayscale value of the first pixel contained in each target local region, determine the first local texture value corresponding to each target local region in the first video frame;

[0261] The first image texture value of the first video frame is determined based on the first local texture value corresponding to each target local region.

[0262] In some embodiments, the first determining unit 403 is further configured to:

[0263] Determine the center first pixel and the edge first pixels surrounding the center first pixel within each local area of ​​the target;

[0264] For each target local region, the gray value of the first pixel at each edge is compared with the gray value of the first pixel at the center to obtain multiple comparison results for each target local region;

[0265] The multiple comparison results of each target local region are binarized to obtain the first local texture value corresponding to each target local region.

[0266] In some embodiments, the first determining unit 403 is further configured to:

[0267] Obtain candidate local texture windows and determine the candidate window range value;

[0268] Obtain the target available resource amount of image computing power resources, and query the window range list based on the target available resource amount to obtain the target window range value. The window range list includes the correlation between different available resource amounts and window range values.

[0269] When the candidate window range value is smaller than the target window range value, the candidate local texture window is adjusted according to the target window range value to obtain the target local texture window;

[0270] When the candidate window range value is greater than or equal to the target window range value, the candidate local texture window is determined as the target local texture window.

[0271] In some embodiments, the video encoding apparatus further includes a processing unit for:

[0272] Obtain the image resolution information corresponding to the first video frame currently to be encoded in the frame sequence to be encoded;

[0273] Based on the image resolution information, the first video frame is downsampled to obtain the downsampled first video frame;

[0274] The first determining unit is further configured to: obtain the grayscale value of each first pixel in the first video frame after downsampling.

[0275] In some embodiments, the adjustment unit 405 is further configured to:

[0276] Get the maximum bitrate value for the video encoder;

[0277] The candidate coding bitrates are obtained by weighting the coding bitrates based on the image texture ratio.

[0278] When the upper limit of the coding bitrate is greater than the candidate coding bitrate, the coding bitrate of the video encoder is adjusted according to the candidate coding bitrate to obtain the target coding bitrate;

[0279] When the upper limit of the encoding bitrate is less than the candidate encoding bitrate, the encoding bitrate of the video encoder is adjusted according to the upper limit of the encoding bitrate to obtain the target encoding bitrate.

[0280] In some embodiments, the adjustment unit 405 is further configured to:

[0281] Obtain the third image texture value of the third video frame that is adjacent to and follows the first video frame in the frame sequence to be encoded;

[0282] When the texture complexity of the third image texture value is detected to be greater than the texture complexity of the first image texture value, the target image texture ratio between the third video frame and the first video frame is determined.

[0283] The target coding bitrate of the video encoder is adjusted according to the texture ratio of the target image to obtain the adjusted target coding bitrate, and the third video frame is encoded according to the adjusted target coding bitrate.

[0284] When the texture complexity of the third image texture value identifier is less than or equal to the texture complexity of the first image texture value identifier, the third video frame is encoded according to the target encoding bitrate.

[0285] In some embodiments, the acquisition unit 401 is further configured to:

[0286] Get the current video bitrate of the target video source;

[0287] The video encoder's bitrate is updated synchronously based on the video source bitrate.

[0288] As described above, this embodiment of the application can obtain the current encoding bitrate of the video encoder in real time. Then, it encodes the video frames in the current frame sequence to be encoded of the target video source according to the encoding bitrate. Furthermore, during the encoding process, the texture complexity between the first video frame to be encoded and the previous adjacent second video frame is compared in real time. The texture complexity represents the richness of the video frame's image content to evaluate the required encoding bitrate. If the texture of the current first video frame is more complex than the previous frame, the image texture ratio between the first and second video frames is determined based on the first and second image texture values. Finally, the encoding bitrate of the video encoder is adjusted according to the image texture ratio. The target encoding bitrate is used to enable the video encoder to encode the first video frame according to the adjusted target encoding bitrate. In this way, compared with the related technology where the video encoder encodes video frames according to the video source bitrate counted over a fixed duration, this application can compare the texture complexity between the preceding and following video frames during the encoding process of the target video source's frame sequence. When the texture of the following video frame is more complex than that of the preceding video frame, the encoding bitrate of the video encoder is adjusted according to the texture ratio between the preceding and following video frames. In this way, the encoding bitrate of the video encoder can be flexibly and finely adjusted at the frame level to ensure the encoding quality of the following video frames by the video encoder and guarantee the video frame image quality of the video encoding.

[0289] The specific implementation of each of the above units can be found in the previous embodiments, and will not be repeated here.

[0290] Referring to Figure 10, which is a structural schematic diagram of a terminal provided in an embodiment of this application, Figure 10 includes a structural block of a portion of the terminal 140 implementing this embodiment. The terminal 140 includes components such as a radio frequency (RF) circuit 510, a memory 515, an input unit 520, a display unit 540, a sensor 550, an audio circuit 560, a wireless fidelity (WiFi) module 570, a processor 580, and a power supply 590. Those skilled in the art will understand that the structure of the terminal 140 shown in Figure 11 does not constitute a limitation on a mobile phone or computer, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0291] The RF circuit 510 can be used to receive and transmit signals during information transmission or calls. In particular, it receives downlink information from the base station and processes it with the processor 580; in addition, it transmits uplink data to the base station.

[0292] The memory 515 can be used to store software programs and modules. The processor 580 executes various terminal functions and video encoding by running the software programs and modules stored in the memory 515.

[0293] The input unit 520 can be used to receive input numeric or character information, and to generate key signal inputs related to the terminal's settings and function control. Specifically, the input unit 520 may include a touch panel 531 and other input devices 532.

[0294] The display unit 540 can be used to display input or provided information, as well as various menus of the terminal. The display unit 540 may include a display panel 541.

[0295] Audio circuit 560, speaker 561, and microphone 562 provide an audio interface.

[0296] In this embodiment, the processor 580 included in the terminal 140 can execute the video encoding method of the previous embodiment.

[0297] The terminal 140 in this application embodiment includes, but is not limited to, mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, and aircraft. This invention can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and assisted driving.

[0298] Figure 11 is a schematic diagram of the server structure provided in an embodiment of this application, which includes a partial structural block of the server 110 implementing this embodiment. The server 110 can vary significantly due to different configurations or performance, and may include one or more central processing units (CPUs) 622 (e.g., one or more processors) and a memory 632, and one or more storage media 630 (e.g., one or more mass storage devices) for storing application programs 642 or data 644. The memory 632 and storage media 630 can be temporary or persistent storage. The program stored in the storage media 630 may include one or more modules (not shown in the figure), each module including a series of instruction operations on the server 600. Furthermore, the central processing unit 622 may be configured to communicate with the storage media 630 and execute the series of instruction operations in the storage media 630 on the server 600.

[0299] Server 600 may also include one or more power supplies 626, one or more wired or wireless network interfaces 650, one or more input / output interfaces 658, and / or one or more operating systems 641, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0300] The central processing unit 622 in server 600 can be used to execute the video encoding method of the embodiments of this application, as follows:

[0301] The process involves: obtaining the current encoding bitrate of the video encoder; obtaining the sequence of frames to be encoded from the target video source, and starting to encode the first video frame in the sequence according to the encoding bitrate using the video encoder; determining the first image texture value of the first video frame currently to be encoded in the sequence, and determining the second image texture value of the second video frame adjacent to the first video frame, where the first video frame is any video frame after the first video frame; when the texture complexity indicated by the first image texture value is greater than the texture complexity indicated by the second image texture value, determining the image texture ratio based on the first and second image texture values; adjusting the encoding bitrate of the video encoder according to the image texture ratio to obtain the target encoding bitrate, and encoding the first video frame according to the target encoding bitrate.

[0302] This application also provides a computer-readable storage medium for storing program code, which is used to execute the video encoding methods of the foregoing embodiments, as follows:

[0303] The process involves: obtaining the current encoding bitrate of the video encoder; obtaining the sequence of frames to be encoded from the target video source, and starting to encode the first video frame in the sequence according to the encoding bitrate using the video encoder; determining the first image texture value of the first video frame currently to be encoded in the sequence, and determining the second image texture value of the second video frame adjacent to the first video frame, where the first video frame is any video frame after the first video frame; when the texture complexity indicated by the first image texture value is greater than the texture complexity indicated by the second image texture value, determining the image texture ratio based on the first and second image texture values; adjusting the encoding bitrate of the video encoder according to the image texture ratio to obtain the target encoding bitrate, and encoding the first video frame according to the target encoding bitrate.

[0304] This application also provides a computer program product, which includes a computer program. A processor of a computer device reads and executes the computer program, causing the computer device to perform the video encoding method described above.

[0305] Furthermore, the terms “comprising” and “including”, and any variations thereof, are intended to cover non-exclusive inclusion, such that a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or apparatus.

[0306] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0307] It should be understood that in the description of the embodiments of this application, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0308] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0309] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0310] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0311] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to related technologies, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0312] It should also be understood that the various implementation methods provided in this application can be combined arbitrarily to achieve different technical effects.

[0313] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0314] The above is a detailed description of the embodiments of this application. However, this application is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A video encoding method, the method being executed by a computer device, the method comprising: Get the current bitrate of the video encoder; The target video source is obtained as a sequence of frames to be encoded, and the first video frame in the sequence is encoded by the video encoder according to the encoding bitrate. Determine the first image texture value of the first video frame currently to be encoded in the frame sequence to be encoded, and determine the second image texture value of the second video frame adjacent to the first video frame, wherein the first video frame is any video frame after the first video frame; When the texture complexity identified by the first image texture value is greater than the texture complexity identified by the second image texture value, the image texture ratio is determined based on the first image texture value and the second image texture value. The encoding bitrate of the video encoder is adjusted according to the image texture ratio to obtain the target encoding bitrate, and the first video frame is encoded according to the target encoding bitrate.

2. The method according to claim 1, further comprising, before determining the first image texture value of the first video frame currently to be encoded in the frame sequence to be encoded: Determine the similarity between the first video frame and the second video frame; Determining the first image texture value of the first video frame currently to be encoded in the frame sequence includes: When the similarity is less than a preset similarity threshold, the first image texture value of the first video frame currently to be encoded in the frame sequence to be encoded is determined.

3. The method according to claim 2, wherein the similarity includes color similarity, and determining the similarity between the first video frame and the second video frame includes: Obtain the first pixel value of each first pixel in the first video frame, and generate a first histogram for the first video frame based on the statistical count of different first pixel values. Obtain the second pixel value of each second pixel point in the second video frame, and generate a second histogram for the second video frame based on the statistical count of different second pixel values; The color similarity between the first video frame and the second video frame is determined based on the difference between the first histogram and the second histogram.

4. The method according to claim 3, wherein determining the color similarity between the first video frame and the second video frame based on the difference between the first histogram and the second histogram comprises: The first statistic of the first video frame in each pixel value dimension is determined based on the first histogram, and the second statistic of the second video frame in each pixel value dimension is determined based on the second histogram; Based on the first and second statistics in each pixel value dimension, determine the difference in the number of pixels between the first video frame and the second video frame in each pixel value dimension; The color similarity between the first video frame and the second video frame is determined based on the difference in the number of pixels in each pixel value dimension.

5. The method according to claim 1, wherein determining the image texture ratio based on the first image texture value and the second image texture value comprises: Obtain the image texture difference between the first image texture value and the second image texture value; When the image texture difference is greater than a preset texture difference threshold, the image texture ratio of the first image texture value and the second image texture value is calculated.

6. The method according to any one of claims 1-5, wherein determining the first image texture value of the first video frame currently to be encoded in the frame sequence to be encoded comprises: Obtain the grayscale value of each first pixel in the first video frame; The first image texture value of the first video frame is determined based on the grayscale value of each of the first pixels.

7. The method according to claim 6, wherein determining the first image texture value of the first video frame based on the grayscale value of each first pixel includes: Get the target local texture window; According to the target local texture window, the first pixel in the first video frame is divided to obtain multiple target local regions after division. Based on the grayscale value of the first pixel contained in each target local region, determine the first local texture value corresponding to each target local region in the first video frame; The first image texture value of the first video frame is determined based on the first local texture value corresponding to each target local region.

8. The method according to claim 7, wherein determining the first local texture value corresponding to each target local region in the first video frame based on the grayscale value of the first pixel contained in each target local region includes: Determine the central first pixel point and the edge first pixels point surrounding the central first pixel point within each target local region; For each target local region, the gray value of each edge first pixel is compared with the gray value of the center first pixel to obtain multiple comparison results for each target local region; The multiple comparison results of each target local region are binarized to obtain the first local texture value corresponding to each target local region.

9. The method according to claim 8, wherein obtaining the target local texture window comprises: Obtain candidate local texture windows and determine the candidate window range value of the candidate local texture windows; Obtain the target available resource amount of image computing power resources, and query the window range list based on the target available resource amount to obtain the target window range value. The window range list includes the correlation between different available resource amounts and window range values. When the candidate window range value is smaller than the target window range value, the candidate local texture window is adjusted according to the target window range value to obtain the target local texture window; When the candidate window range value is greater than or equal to the target window range value, the candidate local texture window is determined as the target local texture window.

10. The method according to any one of claims 6 to 9, wherein before obtaining the grayscale value of each first pixel in the first video frame currently to be encoded in the frame sequence to be encoded, the method further comprises: Obtain the image resolution information corresponding to the first video frame currently to be encoded in the frame sequence to be encoded; Based on the image resolution information, the first video frame is downsampled to obtain the downsampled first video frame. The step of obtaining the grayscale value of each first pixel in the first video frame currently to be encoded in the frame sequence to be encoded includes: Obtain the grayscale value of each first pixel in the first video frame after the downsampling process.

11. The method according to claim 1, wherein adjusting the encoding bitrate of the video encoder according to the image texture ratio to obtain the target encoding bitrate comprises: Obtain the upper limit of the encoding bitrate for the video encoder; The coding bitrate is weighted according to the image texture ratio to obtain the candidate coding bitrate; When the upper limit of the coding bitrate is greater than the candidate coding bitrate, the coding bitrate of the video encoder is adjusted according to the candidate coding bitrate to obtain the target coding bitrate; When the upper limit of the encoding bitrate is less than the candidate encoding bitrate, the encoding bitrate of the video encoder is adjusted according to the upper limit of the encoding bitrate to obtain the target encoding bitrate.

12. The method according to any one of claims 1-11, wherein after encoding the first video frame according to the target coding bitrate, it further comprises: Obtain the third image texture value of the third video frame that is located after and adjacent to the first video frame in the frame sequence to be encoded; When it is detected that the texture complexity of the third image texture value identifier is greater than the texture complexity of the first image texture value identifier, the target image texture ratio between the third video frame and the first video frame is determined; The target encoding bitrate of the video encoder is adjusted according to the target image texture ratio to obtain the adjusted target encoding bitrate, and the third video frame is encoded according to the adjusted target encoding bitrate. When the texture complexity of the third image texture value identifier is less than or equal to the texture complexity of the first image texture value identifier, the third video frame is encoded according to the target encoding bitrate.

13. The method according to any one of claims 1-12, wherein obtaining the current encoding bitrate of the video encoder comprises: Get the current video bitrate of the target video source; The encoding bitrate of the video encoder is updated synchronously based on the video source bitrate.

14. A video encoding apparatus, comprising: The acquisition unit is used to acquire the current encoding bitrate of the video encoder. The encoding unit is used to acquire the sequence of frames to be encoded from the target video source, and to encode the first video frame in the sequence of frames to be encoded by the video encoder according to the encoding bitrate. The first determining unit is used to determine the first image texture value of the first video frame currently to be encoded in the frame sequence to be encoded, and to determine the second image texture value of the second video frame adjacent to the first video frame, wherein the first video frame is any video frame after the first video frame; The second determining unit is configured to determine an image texture ratio based on the first image texture value and the second image texture value when the first image texture value is greater than the second image texture value. An adjustment unit is used to adjust the encoding bitrate of the video encoder according to the image texture ratio to obtain a target encoding bitrate, and to encode the first video frame according to the target encoding bitrate.

15. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the video encoding method according to any one of claims 1 to 13.

16. A storage medium for storing a computer program for performing the method of any one of claims 1-12.

17. A computer program product comprising a computer program, which, when run on a computer, causes the computer to perform the method of any one of claims 1-12.

Images