Video stream coding processing method and device, equipment and medium

Abstract

The invention provides a video stream coding processing method and device, equipment and a medium, and relates to the technical field of video transmission. The method comprises the following steps: determining a to-be-coded image sequence and equipment parameters of user equipment; determining initial coding parameters corresponding to the plurality of quality levels according to the image sequence; according to the equipment parameters, performing dynamic adjustment processing on the plurality of initial coding parameters to obtain a plurality of target coding parameters; and according to the plurality of target coding parameters, coding the image sequence to obtain video streams corresponding to the plurality of quality levels. Through video content and equipment parameters of user equipment, coding parameters are dynamically adjusted, coding efficiency is optimized, a plurality of video streams with different code rates are generated, the code rates and the resolution ratios of the video streams are dynamically adjusted according to network conditions, the continuity and the stability of the video streams are kept, and the problem that the video streams are not uniform in coding and transmission in the coding and transmission process is solved. And the coding and transmission efficiency of the video stream is influenced in the aspects of network bandwidth fluctuation, congestion control, threshold scaling and the like.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of video transmission, and particularly relates to a video stream encoding processing method and device, equipment and a medium. BACKGROUND

[0002] A video is a sequence of continuous images, which is composed of continuous frames, and one frame is one image. A video stream is a transmission mode of video data information, which allows a user to process the received information before receiving complete data information, thereby solving the transmission problem of video data information on a network.

[0003] Due to the high similarity between the continuous frames in the video, in order to facilitate storage and transmission, the original video needs to be encoded and compressed to remove the spatial and temporal redundancy. For video stream transmission, the existing technology encodes the rendered video sequence into a compressed data stream by using a video encoding technology, and then transmits the data stream to a client through a network. The client decodes the data to display the video to the user. This method usually requires huge bandwidth and storage resources when processing large-scale and high-complexity three-dimensional scenes.

[0004] Video stream media transmission technology mainly includes three links: encoding, transmission and decoding. In the existing video stream transmission technology, the video information is compressed, divided into segments and transmitted to a user equipment for decoding. The decoding is performed simultaneously with the playing. This method has the following problems, such as interface lag caused by network bandwidth change, transmission delay and the like, which seriously affect the user's viewing experience. In the encoding and transmission process, problems often occur in network bandwidth fluctuation, congestion control, threshold scaling and the like, which affect the encoding and transmission efficiency of the video stream. SUMMARY

[0005] The application provides a video stream encoding processing method, device, equipment and medium, which solves the problem that the network bandwidth fluctuation, congestion control, threshold scaling and the like in the encoding and transmission process of the existing video stream affect the encoding and transmission efficiency of the video stream.

[0006] In a first aspect, the application provides a video stream encoding processing method, which comprises:

[0007] determining an image sequence to be encoded and a device parameter of a user equipment;

[0008] determining initial encoding parameters corresponding to a plurality of quality levels according to the image sequence;

[0009] dynamically adjusting the plurality of initial encoding parameters according to the device parameter to obtain a plurality of target encoding parameters;

[0010] The image sequence is encoded according to the multiple target encoding parameters to obtain video streams corresponding to multiple quality levels;

[0011] Optionally, determining the initial encoding parameters corresponding to multiple quality levels based on the image sequence includes:

[0012] The image sequence is analyzed and processed to obtain analysis results, which are used to indicate the scene information, complexity, and texture details of the image sequence.

[0013] Based on the scene information, complexity, and texture details of the image sequence, an encoding strategy corresponding to multiple quality levels is determined;

[0014] Based on multiple encoding strategies, the initial encoding parameters corresponding to each quality level are determined, wherein the encoding parameters are used to indicate the setting parameters of the corresponding quantization parameters, frame rate control parameters and keyframe intervals;

[0015] Optionally, the device parameters include: device performance and real-time network status. The step of dynamically adjusting multiple initial encoding parameters based on the device parameters to obtain multiple target encoding parameters includes:

[0016] The range of encoding parameters for the user equipment is determined based on the device performance and the real-time network status.

[0017] For any one of the plurality of initial encoding parameters, determine whether the initial encoding parameter is within the range of the encoding parameters;

[0018] If the initial encoding parameters are not within the range of the encoding parameters, the initial encoding parameters are adjusted to the range of the encoding parameters to obtain the target encoding parameters;

[0019] If the initial encoding parameters are within the range of the encoding parameters, the initial encoding parameters are determined as the target encoding parameters;

[0020] Optionally, the step of encoding the image sequence according to the plurality of target encoding parameters to obtain video streams corresponding to multiple quality levels includes:

[0021] Based on the multiple target encoding parameters, the video streams corresponding to the multiple quality levels are processed in parallel by multiple encoders;

[0022] During the parallel processing of the multiple encoders, a rate control algorithm is used to adjust the real-time rate of the multiple encoders.

[0023] Optionally, after encoding the image sequence according to the plurality of target encoding parameters to obtain video streams corresponding to multiple quality levels, the method further includes:

[0024] Based on the real-time network status, a corresponding transmission strategy is determined, wherein different real-time network statuses correspond to different transmission strategies.

[0025] According to the transmission strategy, a target transmission video stream is determined from the video streams corresponding to the plurality of quality levels, and the target transmission video stream is transmitted to the user equipment;

[0026] Optionally, the method further includes:

[0027] If the real-time network status changes, a new transmission strategy is determined based on the new real-time network status.

[0028] According to the new transmission strategy, a target transmission video stream is determined from the video streams corresponding to the multiple quality levels, and the target transmission video stream is transmitted to the user equipment.

[0029] Optionally, after transmitting the target video stream to the user equipment, the method further includes:

[0030] Monitor whether the target transmitted video stream has a transmission error, and if the target transmitted video stream has a transmission error, determine the erroneous data;

[0031] Based on the error data, an error recovery mechanism is used to transmit the corresponding video stream to the user equipment.

[0032] Secondly, this application provides a video stream encoding processing apparatus, the apparatus comprising:

[0033] The determination module is used to determine the image sequence to be encoded and the device parameters of the user equipment;

[0034] The determining module is further configured to determine initial encoding parameters corresponding to multiple quality levels based on the image sequence;

[0035] The adjustment module is used to dynamically adjust multiple initial encoding parameters according to the device parameters to obtain multiple target encoding parameters;

[0036] The encoding module is used to encode the image sequence according to the multiple target encoding parameters to obtain video streams corresponding to multiple quality levels;

[0037] Optionally, the device further includes: an analysis module;

[0038] The analysis module is used to analyze and process the image sequence to obtain analysis results, which are used to indicate the scene information, complexity, and texture details of the image sequence.

[0039] The determining module is further configured to determine encoding strategies corresponding to multiple quality levels based on scene information, complexity, and texture details of the image sequence.

[0040] The determining module is further configured to determine the initial encoding parameters corresponding to each quality level based on multiple encoding strategies, wherein the encoding parameters are used to indicate the setting parameters of the corresponding quantization parameters, frame rate control parameters and keyframe intervals;

[0041] Optionally, the device further includes: a determination module;

[0042] The determining module is further configured to determine the range of encoding parameters of the user equipment based on the device performance and the real-time network status;

[0043] The determination module is used to determine whether any one of the plurality of initial encoding parameters is within the range of the encoding parameters.

[0044] The adjustment module is further configured to adjust the initial encoding parameters to the range of the encoding parameters when the initial encoding parameters are not within the range of the encoding parameters, thereby obtaining the target encoding parameters;

[0045] The determining module is further configured to determine the initial encoding parameter as the target encoding parameter if the initial encoding parameter is within the range of the encoding parameters;

[0046] Optionally, the device further includes: a processing module;

[0047] The processing module is used to process the video streams corresponding to the multiple quality levels in parallel through multiple encoders based on the multiple target encoding parameters.

[0048] The adjustment module is also used to adjust the real-time bit rate of the multiple encoders using a bit rate control algorithm during the parallel processing of the multiple encoders.

[0049] Optionally, the device further includes: a transmission module;

[0050] The determining module is further configured to determine a corresponding transmission strategy based on the real-time network status, wherein different real-time network statuses correspond to different transmission strategies.

[0051] The determining module is further configured to determine the target transmission video stream from the video streams corresponding to the plurality of quality levels according to the transmission strategy;

[0052] The transmission module is used to transmit the target video stream to the user equipment;

[0053] Optionally, the determining module is further configured to determine a new transmission strategy based on the new real-time network state when the real-time network state changes.

[0054] The determining module is further configured to determine the target transmission video stream from the video streams corresponding to the multiple quality levels according to the new transmission strategy;

[0055] The transmission module is also used to transmit the target video stream to the user equipment;

[0056] Optionally, the device further includes: a monitoring module;

[0057] The monitoring module is used to monitor whether the target transmitted video stream has transmission errors;

[0058] The determining module is further configured to determine erroneous data in the event that an error occurs in the transmission of the target video stream;

[0059] The transmission module is further configured to transmit the transmission video stream corresponding to the error data to the user equipment using an error recovery mechanism based on the error data.

[0060] Thirdly, this application provides a video stream encoding and processing apparatus, comprising:

[0061] processor;

[0062] Memory;

[0063] The memory stores computer-executed instructions;

[0064] The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the video stream coding processing method as described in the first aspect and various possible implementations of the first aspect above.

[0065] Fourthly, embodiments of the present invention provide a readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, performs the video stream encoding processing method as described in the first aspect and various possible implementations thereof.

[0066] This application provides a video stream encoding processing method, apparatus, device, and medium. The method involves determining an image sequence to be encoded and the device parameters of a user device; determining initial encoding parameters corresponding to multiple quality levels based on the image sequence; dynamically adjusting the initial encoding parameters according to the device parameters to obtain multiple target encoding parameters; and encoding the image sequence based on the multiple target encoding parameters to obtain video streams corresponding to multiple quality levels. By dynamically adjusting the encoding parameters based on the video content and the user device's device parameters, encoding efficiency is optimized, generating multiple video streams with different bitrates, and dynamically adjusting the bitrate and resolution of the video streams according to network conditions, maintaining the continuity and stability of the video streams. This solves the problems affecting the encoding and transmission efficiency of video streams during encoding and transmission, such as network bandwidth fluctuations, congestion control, and threshold scaling. Attached Figure Description

[0067] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0068] Figure 1 This is a flowchart illustrating the video stream encoding processing method provided in this application. Figure 1 ;

[0069] Figure 2 This is a flowchart illustrating the video stream encoding processing method provided in this application. Figure 2 ;

[0070] Figure 3 This is a flowchart illustrating the video stream encoding processing method provided in this application. Figure 3 ;

[0071] Figure 4 This is a schematic diagram of the video stream encoding and processing apparatus provided in this application;

[0072] Figure 5 A schematic diagram of the video stream encoding and processing device provided in this application.

[0073] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0074] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0075] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein.

[0076] In this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0077] First, let me explain the terms used in this application:

[0078] Streaming media: Multimedia data is continuously sent to clients by streaming media providers, allowing clients to start playback without downloading the entire multimedia file to their local devices. Streaming media refers to a technology and process that compresses a series of media data, transmits it in segments over the internet, and streams audio and video in real time for viewing. This technology allows data packets to be sent like a flowing stream; without this technology, the entire media file must be downloaded before use. Streaming can transmit live audio and video or videos pre-stored on a server. When viewers watch these audio and video files, the data is immediately played by specific playback software upon arrival at the viewer's computer.

[0079] Quantizer parameter (QP): This parameter indicates the degree of compression of spatial details in an image. A smaller QP value results in finer quantization, higher image quality, and a longer bitstream. If the QP value is small, most details are preserved; if the QP value increases, some details are lost, the bitrate decreases, but image distortion increases, leading to a decrease in quality. Existing bitrate control algorithms primarily output the target bitrate by adjusting the quantization parameter of the discrete cosine transform.

[0080] Frame rate: The frequency at which images appear continuously on a display, measured in frames.

[0081] Keyframes: Keyframes are frames that store the complete image, providing information about the entire image within the video stream. When significant changes occur in the image of a video, such as scene transitions, keyframes must be created to mark these changes.

[0082] Bitrate control algorithms: Bitrate control algorithms are techniques used in video coding to manage the bitrate of a video stream. They aim to dynamically allocate bitrate to different parts of a video sequence based on given bitrate requirements, achieving overall bitrate control and video quality control. These algorithms strive to dynamically adjust the bitrate according to the complexity of the video content to provide high-quality video transmission and storage.

[0083] Constant rate factor (CRF): This refers to the intelligent allocation of bitrate to ensure "a certain quality," including bitrate allocation within the same frame and bitrate allocation between frames. Bitrate allocation within the same frame means allocating more bytes (bits) in the finer details.

[0084] Adaptive Bit Rate (ABR): This refers to the technology that dynamically adjusts the data rate based on network conditions during video streaming.

[0085] User Datagram Protocol (UDP) is a datagram-based protocol that provides packet-switched computer communication in an interconnected computer network environment. It assumes IP as the underlying protocol and operates at the transport layer according to the Open Systems Interconnection (OSI) model. UDP provides applications with a process for sending messages to other programs with minimal protocol mechanisms. This protocol is transaction-oriented and does not guarantee delivery or deduplication.

[0086] Transmission Control Protocol (TCP) is a connection-oriented, reliable, byte-stream-based transport layer communication protocol. TCP is designed to adapt to layered protocol hierarchies that support multiple network applications. Pairs of processes on host computers connected to different but interconnected computer networks rely on TCP for reliable communication services.

[0087] Forward Error Correction (FEC), or channel coding, is a technique for controlling transmission errors in a one-way communication system. It reduces the bit error rate by sending additional information along with the data to perform error recovery. FEC is further divided into in-band FEC and out-of-band FEC.

[0088] Video streaming: Video streaming is a method of transmitting video data that allows users to process received information before receiving the complete data, thus achieving a simultaneous reception and playback effect. This greatly improves the transmission of video data over the network. Users do not need to wait for the entire file to download before starting to watch, and they will not feel that the file has not been completely transmitted during playback. Video streaming technology breaks video files into smaller data packets, enabling them to be easily transmitted over the internet and played in real time at the receiving end.

[0089] Video streaming transmission technology mainly consists of three stages: encoding, transmission, and decoding. In existing video streaming technologies, video information is compressed, segmented, and transmitted over the network to the user's device for decoding, with playback occurring simultaneously. This method has various problems. For example, fluctuations in network bandwidth can cause interface stuttering and transmission delays, severely impacting the user's viewing experience. Furthermore, during encoding and transmission, fluctuations in network bandwidth, congestion control, and threshold scaling frequently cause issues, affecting the encoding and transmission efficiency of the video stream.

[0090] To address the aforementioned problems, this application provides a video stream encoding processing method. This method analyzes the image sequence corresponding to the rendered video to determine the encoding parameters of the video stream. Based on the user's device parameters and network conditions, the encoding parameters are further adjusted. The image sequence is then encoded using these parameters to generate video streams with multiple quality levels. The encoding strategy is optimized and adjusted based on the user's network bandwidth, device performance, and video content to improve encoding efficiency. Furthermore, the encoded video streams can adapt to different users' network conditions and device performance.

[0091] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0092] Figure 1 This is a flowchart illustrating the video stream encoding processing method provided in the embodiments of this application. Figure 1 .like Figure 1As shown in the figure, the video stream encoding processing method illustrated in this embodiment includes:

[0093] S101: Determine the image sequence to be encoded and the device parameters of the user equipment.

[0094] The image sequence to be encoded refers to the consecutive frames that need to be transmitted as video. Before determining the image sequence, the device can receive an access request from the user equipment, which can be used to request access to relevant video information. The device can analyze and determine the 3D information that the user wants to access based on the user's access request, and then render and process it to obtain the image sequence corresponding to the user's access request, which is then transmitted to the user equipment via a video stream. The device parameters of the user equipment refer to the user equipment's network information, device performance, or hardware configuration, etc.

[0095] Understandably, it's helpful to first determine the correlation between the images in the image sequence. For example, in video coding, consecutive frames typically exhibit high temporal correlation, which can improve coding efficiency. Furthermore, the choice of image sequence can vary depending on the application scenario. For instance, video surveillance might select consecutive frames containing moving objects, while static image compression might prioritize spatial correlation. After selecting the image sequence, its quality can be assessed to ensure it meets coding requirements. This can be done by calculating metrics such as image sharpness, contrast, and color saturation.

[0096] S102: Determine the initial coding parameters corresponding to multiple quality levels based on the image sequence.

[0097] Different quality levels correspond to different bitrates and resolutions in video streams. Encoding parameters refer to data related to image quality, file size, encoding and decoding speeds, such as quantization parameters, bitrate, frame rate, and resolution. After determining the image sequence, preprocessing can be performed, including noise reduction, color correction, and resolution adjustment. Noise reduction aims to reduce or eliminate noise in the image, improving image clarity and signal-to-noise ratio. Color correction adjusts image colors to make them more accurate and consistent, eliminating color deviations caused by device differences, lighting conditions, etc. Resolution adjustment changes image size and clarity; depending on the application scenario, the image can be adjusted to different resolutions. Through preprocessing operations such as noise reduction, color correction, and resolution adjustment, images can be made clearer, more accurate, and more consistent.

[0098] Understandably, depending on the encoding requirements, a suitable encoding scheme can be selected based on the characteristics of the image sequence and the application scenario. For example, for image sequences containing a large amount of motion information, an encoding scheme that supports motion estimation and compensation can be selected; for static images or image sequences with little change, an encoding scheme that focuses more on spatial correlation can be selected.

[0099] Specifically, there are multiple quality levels, such as high quality, medium quality, and low quality. High quality refers to image quality that provides higher image sharpness and detail retention, but with larger file sizes and longer encoding and decoding times; medium quality strikes a balance between image quality and file size, suitable for most application scenarios; low quality refers to image quality with smaller file sizes but significantly reduced image quality, suitable for scenarios with limited network transmission speeds or storage space.

[0100] In one possible implementation, a set of initial encoding parameters can be preset according to the basic requirements of the application scenario. These parameters may include bit rate, frame rate, resolution, encoding mode, keyframe interval, etc. When setting encoding parameters, the image resolution can be considered. For example, high-resolution images usually require higher bit rates and finer quantization parameters to maintain image quality. The complexity of the image content also affects the setting of encoding parameters. For example, for images containing a lot of detail and texture, lower quantization parameters and higher bit rates may be needed to avoid image distortion. The decoding capability and compatibility of the target device are also factors to be considered when setting encoding parameters. For example, for mobile devices or low-performance user devices, lower encoding complexity and smaller file size may be required. Due to the special characteristics of different encoding algorithms and image sequences, adjustments and optimizations can be made according to specific circumstances, and this application does not impose any restrictions on this.

[0101] S103: Based on the device parameters, multiple initial encoding parameters are dynamically adjusted to obtain multiple target encoding parameters.

[0102] This involves acquiring real-time user equipment parameters through relevant communication interfaces or protocols. These parameters may include network speed, latency, signal strength, CPU processing power, memory usage, storage space, and screen resolution. Analyzing these collected parameters can help assess the overall capabilities and current status of the device. For example, it can help determine whether the device can support high-quality video encoding, whether battery conservation is necessary, and whether it is limited by network bandwidth or latency.

[0103] Understandably, based on the analysis results of device parameters, the initial encoding parameters can be dynamically adjusted in real time. Specific adjustment strategies include: adjusting according to network conditions; if the network speed is low or the latency is high, the bit rate and frame rate can be reduced to reduce data transmission pressure and improve smoothness; adjusting according to device performance; if the device has strong CPU processing power and sufficient memory, the resolution and frame rate can be increased; and adjusting according to hardware configuration; if the device screen resolution is high, corresponding high-resolution video can be output; if the device storage space is limited, the size of the encoded file needs to be controlled.

[0104] The adjusted encoding parameters can be verified during the encoding process to provide optimal performance under the current device conditions. During the configuration and dynamic adjustment of encoding parameters, deep learning can be used to continuously optimize encoding efficiency. Through training and learning, the optimal encoding strategy can be selected at each stage. For example, the encoding mode selection, motion estimation, and transform coefficient quantization can be optimized. While ensuring video quality, processing efficiency can be improved and the amount of data in the encoded video stream can be reduced.

[0105] S104: The image sequence is encoded according to the multiple target encoding parameters to obtain video streams corresponding to multiple quality levels.

[0106] This process involves encoding an image sequence using multiple predetermined target encoding parameters to generate video streams corresponding to various quality levels. The video content can be encoded at different resolutions, bit rates, or frame rates to accommodate different users' device capabilities and network conditions. The different quality levels of the video stream can be determined based on the number and range of target encoding parameters.

[0107] Understandably, for each quality level, corresponding target encoding parameters can be applied for encoding. The desired video quality and compression efficiency can be achieved by setting specific encoder parameters, such as QP, GOP structure, number of reference frames, prediction mode, etc.

[0108] Specifically, for resolution-level encoding, it ensures the encoder uses the correct output resolution to generate the video stream; for bitrate-level encoding, the encoder's bitrate control parameters can be adjusted to match the target bitrate; and for frame rate-level encoding, the encoder's frame rate parameters can be set to ensure the output video stream has the correct frame rate. Using a video encoder to encode image sequences, based on the target encoding parameters, the encoder can compress the image sequences and generate video streams of multiple quality levels. These generated video streams can be saved in different files or transmitted to the user as a stream.

[0109] In one possible implementation, the encoding parameters can be optimized and adjusted based on the verification results. This could include fine-tuning quantization parameters, changing encoder settings, or trying different encoding strategies to find the optimal combination of encoding parameters to achieve the best video quality and compression ratio at a given quality level.

[0110] The video stream encoding processing method provided in this embodiment determines the image sequence to be encoded and the device parameters of the user equipment; based on the image sequence, it determines initial encoding parameters corresponding to multiple quality levels; based on the device parameters, it dynamically adjusts the multiple initial encoding parameters to obtain multiple target encoding parameters; based on the multiple target encoding parameters, it encodes the image sequence to obtain video streams corresponding to multiple quality levels. By dynamically adjusting the encoding parameters based on the video content and the device parameters of the user equipment, it optimizes encoding efficiency, generates multiple video streams with different bitrates, and dynamically adjusts the bitrate and resolution of the video streams according to network conditions, maintaining the continuity and stability of the video streams.

[0111] Figure 2 This is a flowchart illustrating the video stream encoding processing method provided in the embodiments of this application. Figure 1 This embodiment is... Figure 2 Based on the embodiments, the video stream encoding and processing method is described in detail. For example... Figure 3 As shown in the figure, the video stream encoding processing method illustrated in this embodiment includes:

[0112] S201: Determine the image sequence to be encoded and the device parameters of the user equipment.

[0113] Step S201 is similar to step S101 above, and will not be described again here.

[0114] S202: The image sequence is analyzed and processed to obtain analysis results, which are used to indicate the scene information, complexity, and texture details of the image sequence.

[0115] The process involves analyzing and processing image sequences to obtain results, including scene information, complexity, and texture details. Image preprocessing involves extracting consecutive image frames from video files as the analysis object and applying denoising algorithms to reduce noise and improve the accuracy of subsequent analysis. Depending on the needs of the image, it can be converted from a color space to a more suitable one for analysis.

[0116] Understandably, scene information analysis can involve selecting the most representative frames as keyframes within each scene. These frames summarize the main content of the scene; machine learning or deep learning models can be used to classify the keyframes and identify the scene type. Complexity analysis can be performed using edge detection, texture analysis, etc., to determine the complexity of the image sequence, analyze the texture features of the image, and calculate the image's information entropy. Texture detail analysis can be conducted through multi-scale analysis, frequency domain analysis, and feature extraction to obtain the texture details of the image sequence. Integrating the analysis results of scene information, complexity, and texture details forms a comprehensive description of the image sequence.

[0117] S203: Determine the encoding strategies corresponding to multiple quality levels based on the scene information, complexity, and texture details of the image sequence.

[0118] The encoding strategy primarily aims to reduce redundant information in video data, thereby improving compression efficiency and transmission quality. Scene information analysis determines whether the current scene is static or dynamic. For static scenes, the frame rate can be appropriately reduced to decrease redundant information; for dynamic scenes, the frame rate can be maintained or increased to capture motion details. Furthermore, the strategy can be tailored to factors such as lighting or scene information. For example, scenes with significant lighting changes may require a higher bit rate to maintain detail and color accuracy; finer encoding parameters can be used at scene transitions to reduce visual discontinuities during transitions.

[0119] Understandably, through complexity analysis, for high-complexity scenes—those containing a lot of detail, fast motion, or high contrast—a lower QP can be used to improve encoding quality and reduce distortion. For low-complexity scenes—those that are relatively simple and don't change much—the quantization parameter can be appropriately increased to save bitrate. The image can be divided into regions of different complexities, and different encoding parameters can be applied to each region.

[0120] By analyzing texture details, higher bitrates and finer quantization settings can be set for areas with rich texture details to maintain the clarity and layering of the texture; while for relatively smooth areas with less texture, lower bitrates and coarser quantization settings can be used to reduce the amount of data.

[0121] Specifically, based on the above analysis, a corresponding encoding strategy can be determined for each quality level. This strategy could include: Resolution: Higher resolution for high-quality levels and lower resolution for low-quality levels; Bitrate: Higher bitrate for high-quality levels to ensure detail and color accuracy, and lower bitrate for low-quality levels to save bandwidth or storage space; Frame Rate: Adjusting the frame rate dynamically according to the scene, maintaining a higher frame rate for dynamic scenes and appropriately lowering it for static scenes; QP: Adjusting the quantization parameters according to complexity and texture detail, using a lower QP for high-complexity or texture-rich areas and a higher QP for low-complexity or smooth areas; Encoding Mode: Selecting an encoding mode suitable for the current content, such as the allocation ratio of I-frames, P-frames, and B-frames, and whether to use techniques such as variable block size and skipped frames.

[0122] S204: Based on multiple encoding strategies, determine the initial encoding parameters corresponding to each quality level, wherein the encoding parameters are used to indicate the setting parameters of the corresponding quantization parameters, frame rate control parameters, and keyframe intervals.

[0123] In this process, by determining the corresponding encoding strategy for each quality level, the initial encoding parameters for each quality level can be determined. These encoding parameters may include, for example, quantization parameters, frame rate control parameters, and keyframe interval setting parameters.

[0124] Understandably, multiple quality levels can be defined as high, medium, and low. For high quality, to maintain image detail and sharpness, a lower quantization parameter should be used. A lower QP value means less quantization error, thus preserving more image information, but it also increases the amount of encoded data. Medium quality can use a moderate QP value to find a balance between image quality and coding efficiency. Low quality, to save bandwidth or storage space, should use a higher quantization parameter. A higher QP value leads to more quantization error, resulting in some loss of image detail, but significantly reducing the amount of encoded data.

[0125] The frame rate control parameters can be determined according to different scenarios: for scenarios with fast motion or high dynamic changes, a higher frame rate can be maintained to ensure smooth motion. At the high quality level, the frame rate can be further increased to provide a smoother visual experience. For relatively static scenarios, the frame rate can be appropriately reduced at the low quality level to reduce the amount of data, while the high quality level maintains a standard or slightly higher frame rate to maintain image continuity.

[0126] One possible implementation is to set an adaptive frame rate, which dynamically adjusts the frame rate based on the complexity of the scene or the motion conditions to achieve better encoding results.

[0127] Specifically, regarding the keyframe interval setting, a shorter keyframe interval can be set at high quality levels, which helps reduce the cumulative error caused by prediction errors, especially in complex scenes or fast-moving scenarios. For medium and low quality levels, the keyframe interval can be appropriately increased as the quality level decreases to reduce the amount of encoded data. It should be noted that an excessively long keyframe interval may cause noticeable visual jumps when the scene changes or motion suddenly increases. Therefore, the keyframe interval setting can be refined using machine learning to achieve smooth video streaming. Simultaneously, scene change detection algorithms can be combined to automatically insert keyframes when scene transitions are detected, ensuring continuity between different scenes.

[0128] S205: Determine the range of encoding parameters for the user equipment based on the device performance and the real-time network status.

[0129] The encoding parameter range of a user equipment (UE) refers to the range of encoding parameters corresponding to the video stream that the UE can receive. Evaluating the UE's performance determines the hardware capabilities of its processor, memory, and other components. Real-time network status analysis helps determine the optimal quality target video stream that the user can receive.

[0130] Understandably, assessing a user's device's CPU or GPU performance determines its ability to decode complex video streams; high-performance devices can support higher resolutions, frame rates, and more complex encoding settings. Determining the device's available memory and storage space ensures that the encoded video stream doesn't consume excessive resources, thus impacting the device's overall performance. Analyzing the device's screen resolution and display capabilities helps determine the optimal encoding resolution and color depth.

[0131] Real-time measurement of user device network bandwidth, including upload and download speeds, can be achieved, for example, through small packet exchange with the server or by using network speed testing services. Network latency and packet loss rate are also monitored to determine suitable video streaming protocols and buffering strategies. In networks with high latency or high packet loss rates, larger buffers and more robust error correction mechanisms may be required. Simultaneously, the type of network the user is currently connected to is identified, such as Wi-Fi, 4G / 5G mobile networks, etc. Different network types have different characteristics and limitations, and different encoding requirements can be determined based on the network type.

[0132] Specifically, based on the device's screen characteristics and network bandwidth, select an appropriate resolution range. For example, in situations with limited bandwidth, a lower resolution can be chosen to reduce data transmission. Determine an appropriate frame rate range based on the device's processing power and network latency. For instance, a higher frame rate can provide a smoother video experience but also increases the amount of encoded data and transmission latency. Based on network bandwidth and the user's video quality requirements, select an appropriate QP value range. Lower QP values ​​provide better video quality but correspondingly increase data volume, while higher QP values ​​have the opposite effect. Set an appropriate keyframe interval based on the network packet loss rate and the dynamic nature of the video content. Shorter intervals can improve error recovery capabilities but also increase data volume.

[0133] S206: For any one of the plurality of initial encoding parameters, determine whether the initial encoding parameter is within the range of the encoding parameters.

[0134] The range of encoding parameters may include, for example, the resolution range, the frame rate range, the quantization parameter (QP) range, and the keyframe interval range.

[0135] Understandably, from the given set of initial encoding parameters, a single initial encoding parameter that needs to be judged is extracted. The extracted initial encoding parameter is compared with the corresponding encoding parameter range, that is, checking whether the initial resolution falls within the allowed resolution range; whether the initial frame rate is within the set frame rate range; whether the initial QP value is within the allowed QP value range; and whether the initial keyframe interval conforms to the set interval range.

[0136] Specifically, based on the comparison results, it is determined whether the initial encoding parameter is within the range of encoding parameters: if it is within the range, it indicates that the initial encoding parameter is appropriate and can continue to be used or used as a starting point for further optimization; if it is not within the range, it indicates that the initial encoding parameter may not be suitable for the current device performance or network status and needs to be adjusted.

[0137] S207: If the initial encoding parameters are not within the range of the encoding parameters, adjust the initial encoding parameters to the range of the encoding parameters to obtain the target encoding parameters.

[0138] If an initial encoding parameter is outside the allowed range, it should be adjusted based on device performance and real-time network conditions until it meets the requirements of the encoding parameter range. This judgment and adjustment process can be repeated for multiple initial encoding parameters until all parameters meet the requirements of the encoding parameter range.

[0139] Understandably, machine learning algorithms can be used to monitor the user's network status and device performance, dynamically adjusting the aforementioned encoding parameters based on factors such as video content complexity and network conditions. For example, for dynamic scenes, the QP can be reduced to maintain image quality; for static scenes, the QP can be appropriately increased to reduce the amount of encoded data. Scene-adaptive frame rate control techniques can be employed, adjusting the frame rate according to the speed of scene changes to balance video quality and transmission bandwidth. Regarding the quantization parameter (QP), different quantization step sizes can be set for video streams with different bitrates to control compression ratio and image quality; different frame rates can be set for video streams with different bitrates to adapt to different dynamic scenes; and the keyframe interval can be adjusted to optimize encoding efficiency and decoding performance.

[0140] S208: If the initial encoding parameters are within the range of the encoding parameters, the initial encoding parameters are determined as the target encoding parameters.

[0141] S209: Based on the multiple target encoding parameters, the video streams corresponding to the multiple quality levels are processed in parallel by multiple encoders.

[0142] This can be achieved using a multi-bitrate video encoder, such as H.264 multi-bitrate encoding mode or H.265 layered encoding technology, to generate multiple video streams of different quality levels, each with its specific bitrate and resolution. The encoder can read raw video data from the video source. If the video source supports multi-output or splitting, different parts can be directly assigned to different encoders, or the video data can be copied or split into multiple streams and then sent to different encoders respectively. Multiple encoders are started to process the video streams of different quality levels in parallel. Each encoder encodes the video stream according to the assigned target encoding parameters, generating encoded video data of the corresponding quality level.

[0143] Understandably, encoder performance metrics such as encoding speed, CPU / GPU utilization, and memory usage can be monitored in real time to ensure stable system operation. The quality of the encoded video streams should be checked periodically to ensure they meet expected encoding parameters and quality standards. Encoder resource allocation can be dynamically adjusted based on system load and video stream demands to optimize performance and resource utilization. The encoded video streams are then output to designated locations or devices, such as local storage, web servers, or streaming media distribution networks. Each quality level of video stream can be stored and distributed separately, allowing for selection of appropriate video quality for playback based on the user's device performance and network conditions.

[0144] Furthermore, as the system scales up and demands increase, encoder resources can be expanded, encoding strategies optimized, or new encoding technologies introduced to accommodate higher processing capabilities and more complex video content. By processing multiple quality levels of video streams in parallel, the efficiency and flexibility of video encoding can be significantly improved, meeting the viewing needs of different users in different scenarios.

[0145] S210: During the parallel processing of the multiple encoders, a rate control algorithm is used to adjust the real-time rate of the multiple encoders.

[0146] During the encoding process, bitrate control algorithms, such as CRF or ABR, can be used to dynamically adjust the bitrate allocation to ensure the quality and transmission efficiency of the video stream.

[0147] The video stream encoding processing method provided in this embodiment analyzes and processes image sequences to obtain scene information, complexity, and texture details, thereby determining encoding parameters for multiple quality levels. These parameters are then adjusted in real-time based on the user's device performance and real-time network conditions to ensure they remain within a specified range. Finally, based on the target encoding parameters, multiple encoders process the video streams corresponding to the multiple quality levels in parallel. By monitoring environmental changes in real-time and automatically adjusting encoding, the method ensures that the encoding parameter settings are compatible with both the user's device processing capabilities and real-time network conditions, thus providing optimal video encoding and transmission performance. Furthermore, by processing video streams of multiple quality levels in parallel, the efficiency and flexibility of video encoding are significantly improved, meeting the viewing needs of different users in various scenarios.

[0148] Figure 3 This is a flowchart illustrating the video stream encoding processing method provided in the embodiments of this application. Figure 1 This embodiment is... Figure 2 and Figure 3 Based on the embodiments, the process of transmitting video streams to user equipment after obtaining video streams corresponding to multiple quality levels is described in detail. For example... Figure 4 As shown in the figure, the video stream encoding processing method illustrated in this embodiment includes:

[0149] S301: Determine the corresponding transmission strategy based on the real-time network status, wherein different real-time network statuses correspond to different transmission strategies.

[0150] In network communication, transmission strategy refers to the data transmission scheme formulated based on various factors such as real-time network conditions, user needs, and equipment capabilities. Different real-time network conditions, such as bandwidth, latency, and packet loss rate, can lead to different transmission strategies to optimize the transmission effect of video streams.

[0151] Understandably, monitoring network status can be achieved by monitoring metrics such as bandwidth, latency, or packet loss rate to determine the real-time state of the network. Real-time measurement of upload and download speeds reveals the current network's transmission capacity; measuring the round-trip time of data packets determines latency; and recording the proportion of packets lost during transmission reveals the network's stability and reliability. Monitored network status data can be compared to preset thresholds or standards to analyze the network's condition and trends. For example, if bandwidth falls below a certain threshold, the network may be considered limited; high latency or a high packet loss rate may indicate poor network quality.

[0152] Specifically, based on the network conditions analyzed, corresponding transmission strategies are determined. Different network conditions allow for different transmission strategies. For example, when bandwidth is limited, the video stream bitrate can be reduced to decrease data transmission volume and avoid network congestion and buffering delays; when bandwidth is sufficient, the video stream bitrate can be increased to provide better video quality and viewing experience. When network quality is poor, the video resolution can be reduced to alleviate network load; when network quality is good, the video resolution can be maintained or increased to maintain or improve viewing quality. In high-latency or high-packet-loss network environments, the frame rate can be appropriately reduced to decrease latency caused by retransmissions and buffering; when network quality is stable, the frame rate can be maintained or increased to provide a smoother video experience.

[0153] It's important to note that the size and strategy of the video stream buffer can be adjusted based on network latency and bandwidth fluctuations to balance latency and smoothness. During transmission, a suitable transmission protocol can be selected based on the current network conditions. For example, in bandwidth-limited or high-latency networks, UDP-based transmission protocols such as RTP / RTCP can be used. In environments with ample bandwidth and stable networks, TCP-based protocols such as HTTP Live Streaming and HLS may be preferred. Using secure transmission protocols during video stream transmission ensures the security and integrity of data transmission. Furthermore, encryption and authentication of the video stream can ensure that the video content is not tampered with during transmission, protecting copyright and user privacy.

[0154] S302: According to the transmission strategy, determine the target transmission video stream from the video streams corresponding to the plurality of quality levels, and transmit the target transmission video stream to the user equipment.

[0155] Transmission strategies may include, for example, adaptive bitrate, multipath transmission, flow control, data compression, and priority sorting. ABR can adjust the video stream bitrate in real time based on the user's network bandwidth. When network bandwidth is high, a higher bitrate video stream is provided to improve video quality; when network bandwidth is limited, the bitrate is reduced to decrease buffering time and improve smoothness.

[0156] Understandably, multiple network paths can be used to transmit data simultaneously to improve transmission reliability and throughput. When a path fails or becomes congested, it can automatically switch to another path. Network congestion can also be avoided by adjusting the sending rate at the sending end. When the receiving end's buffer is full, it sends a signal to the sending end to stop transmitting; when there is free space in the buffer, it sends a signal to continue transmitting. Congestion control algorithms can include, for example, TCP's slow start, congestion avoidance, fast retransmission, and fast recovery.

[0157] Specifically, the video stream can be prioritized based on the different scenes within it, with higher-priority scenes transmitted first. This ensures timely delivery of critical data and improves overall system performance. Furthermore, large data blocks can be segmented into smaller data packets for transmission. The receiving end then reassembles these packets into the original data, helping to adapt to the maximum transmission unit limitations of different network paths and enhancing transmission flexibility.

[0158] It should be noted that the most suitable transmission protocol for the current scenario can be selected based on factors such as real-time network status, device capabilities, and user needs. For example, UDP protocol may be selected in scenarios with high real-time requirements, while TCP protocol may be selected in scenarios requiring high reliability.

[0159] S303: If the real-time network status changes, a new transmission strategy shall be determined based on the new real-time network status.

[0160] During transmission, the system can continuously monitor the network status and adjust the transmission strategy or switch video streams of different video qualities as needed to ensure the continuity and stability of the video stream.

[0161] S304: According to the new transmission strategy, determine the target transmission video stream from the video streams corresponding to the plurality of quality levels, and transmit the target transmission video stream to the user equipment.

[0162] The transmission strategy can be dynamically adjusted based on real-time network conditions. During video transmission, adaptive bitrate streaming technologies, such as HTTP Live Streaming or Dynamic Adaptive Streaming over HTTP, can be used. The system then reassesses the user's network conditions and device performance, dynamically selecting the most suitable video stream based on the assessment. The system has a real-time monitoring and feedback mechanism to respond to changes in the user's device's network status and adjust the transmission strategy to maintain the smoothness and quality of the video stream.

[0163] Understandably, an adaptive transmission mechanism can be used to dynamically adjust the transmission strategy based on real-time monitoring of the user's network conditions, seamlessly switching to the most suitable bitrate video stream version, and selecting the most suitable transmission protocol supported by the current user based on the network environment, such as HTTP, UDP, and QUIC. Simultaneously, multi-path transmission technology can be supported to transmit data in parallel through multiple network paths, thereby improving transmission stability and reliability.

[0164] The goal of adjusting transmission strategies is to optimize user experience. When determining and adjusting transmission strategies, factors such as user viewing needs, device capabilities, network conditions, and service provider strategies can be comprehensively considered to provide the best viewing experience.

[0165] S305: Monitor whether the target transmitted video stream has a transmission error, and if the target transmitted video stream has a transmission error, determine the erroneous data.

[0166] Transmission errors refer to situations such as data errors, corruption, video delays, timestamp errors, or data packet loss that occur during transmission. Examples of transmission errors include: data packet loss causing stuttering, skipping, or missing frames in the video stream; data corruption potentially resulting in noise, pixelation, or complete distortion in the video; delays and jitter causing delayed playback, choppy visuals, or skipping; incorrect sequence numbers or timestamps preventing the receiving end from correctly reassembling the video stream, potentially leading to garbled visuals or playback failure; and protocol errors causing data transmission failures, resulting in the inability to receive or play the video stream.

[0167] Understandably, transmission errors can be identified using error detection algorithms. When an error is detected, information such as the error type, time, and location can be recorded in an error log. Detailed analysis of the error log can pinpoint the specific data packet or segment that caused the error, including information such as the sequence number, timestamp, and specific location of the checksum failure. If the video stream is transmitted in multiple segments, independent error detection can be performed on each segment. Once a segment is identified as containing erroneous data, it can be retransmitted or undergo error recovery processing.

[0168] S306: Based on the error data, an error recovery mechanism is used to transmit the video stream corresponding to the error data to the user equipment.

[0169] The appropriate error recovery mechanism can be selected based on the type and nature of the error. For example, FEC (Fault-Error Recovery) technology can be used to recover a small number of lost data packets; retransmission can be requested for a large number of errors, lost keyframes, or corrupted data.

[0170] Understandably, adding redundant information to transmitted data packets allows the receiving end to detect and correct a certain number of errors, reducing retransmission needs due to packet loss and lowering latency. If it is determined that some data packets cannot be recovered through the error recovery mechanism, the system will send a retransmission request to the sending end, requesting the retransmission of lost or corrupted data packets. When the receiving end detects packet loss, it can send a retransmission request to the sending end, which will then retransmit the lost data packets upon receiving the request.

[0171] Specifically, when using FEC, the receiver can attempt to correct erroneous data using additional checksum information. If successful, retransmission is unnecessary; if it fails, other recovery measures or retransmission can be considered. During decoding, when erroneous data is detected, error concealment techniques can be used to smoothly transition or interpolate lost data.

[0172] It should be noted that during error recovery or retransmission, the transmission strategy can be dynamically adjusted, such as reducing the bit rate, increasing the buffer size, or changing the transmission path, to adapt to the current network conditions and reduce the possibility of future transmission errors.

[0173] The video stream encoding and processing method provided in this embodiment generates a corresponding video stream transmission strategy based on the real-time network status of the user equipment. Different transmission strategies are used to optimize the transmission effect of the video stream. By continuously monitoring the network status, the transmission strategy is adjusted in real time or different video quality video streams are switched to ensure the continuity and stability of the video stream. During transmission, potential transmission errors are monitored, erroneous data is identified, and data recovery is performed through error recovery or retransmission mechanisms. Errors during video stream transmission are handled in real time to ensure that users can receive high-quality, uninterrupted video content.

[0174] Figure 4 This is a schematic diagram of the video stream encoding and processing apparatus provided in this application. Figure 5 As shown, the video stream encoding processing apparatus 400 provided in this embodiment includes:

[0175] The determination module 401 is used to determine the image sequence to be encoded and the device parameters of the user equipment;

[0176] The determining module 401 is further configured to determine initial encoding parameters corresponding to multiple quality levels based on the image sequence;

[0177] The adjustment module 402 is used to dynamically adjust multiple initial encoding parameters according to the device parameters to obtain multiple target encoding parameters;

[0178] The encoding module 403 is used to encode the image sequence according to the multiple target encoding parameters to obtain video streams corresponding to multiple quality levels;

[0179] Optionally, the device further includes: an analysis module 404;

[0180] The analysis module 404 is used to analyze and process the image sequence to obtain analysis results, which are used to indicate the scene information, complexity, and texture details of the image sequence.

[0181] The determining module 401 is further configured to determine the encoding strategies corresponding to multiple quality levels based on the scene information, complexity, and texture details of the image sequence.

[0182] The determining module 401 is further configured to determine the initial encoding parameters corresponding to each quality level based on multiple encoding strategies, wherein the encoding parameters are used to indicate the setting parameters of the corresponding quantization parameters, frame rate control parameters and keyframe intervals;

[0183] Optionally, the device further includes: a determination module 405;

[0184] The determining module 401 is further configured to determine the encoding parameter range of the user equipment based on the device performance and the real-time network status;

[0185] The judgment module 405 is used to determine whether any one of the plurality of initial encoding parameters is within the range of the encoding parameters.

[0186] The adjustment module 402 is further configured to adjust the initial encoding parameters to the range of the encoding parameters when the initial encoding parameters are not within the range of the encoding parameters, so as to obtain the target encoding parameters;

[0187] The determining module 401 is further configured to determine the initial encoding parameter as the target encoding parameter when the initial encoding parameter is within the range of the encoding parameter;

[0188] Optionally, the device further includes: a processing module 406;

[0189] The processing module 406 is used to process the video streams corresponding to the multiple quality levels in parallel using multiple encoders based on the multiple target encoding parameters.

[0190] The adjustment module 402 is also used to adjust the real-time bit rate of the multiple encoders using a bit rate control algorithm during the parallel processing of the multiple encoders.

[0191] Optionally, the device further includes: a transmission module 407;

[0192] The determining module 401 is further configured to determine a corresponding transmission strategy based on the real-time network status, wherein different real-time network statuses correspond to different transmission strategies.

[0193] The determining module 401 is further configured to determine the target transmission video stream from the video streams corresponding to the plurality of quality levels according to the transmission strategy;

[0194] The transmission module 407 is used to transmit the target video stream to the user equipment;

[0195] Optionally, the determining module 401 is further configured to determine a new transmission strategy based on the new real-time network state when the real-time network state changes.

[0196] The determining module 401 is further configured to determine the target transmission video stream from the video streams corresponding to the plurality of quality levels according to the new transmission strategy;

[0197] The transmission module 407 is also used to transmit the target video stream to the user equipment;

[0198] Optionally, the device further includes: a monitoring module 408;

[0199] The monitoring module 408 is used to monitor whether the target transmitted video stream has a transmission error;

[0200] The determining module 401 is further configured to determine erroneous data when an error occurs in the transmission of the target video stream;

[0201] The transmission module 407 is further configured to transmit the transmission video stream corresponding to the error data to the user equipment using an error recovery mechanism based on the error data.

[0202] Figure 5 This is a schematic diagram of the video stream encoding and processing device provided in this application. ​ As shown, this application provides a video stream encoding processing device 500, which includes: a receiver 501, a transmitter 502, a processor 503, and a memory 504.

[0203] Receiver 501 is used to receive instructions and data;

[0204] Transmitter 502 is used to send commands and data;

[0205] Memory 504 is used to store instructions executed by the computer;

[0206] The processor 503 is used to execute computer execution instructions stored in the memory 504 to implement the various steps performed by the various execution entities of the video stream encoding processing method in the above embodiments. For details, please refer to the relevant descriptions in the foregoing embodiments of the video stream encoding processing method.

[0207] Alternatively, the memory 504 can be either standalone or integrated with the processor 503.

[0208] When the memory 504 is set up independently, the electronic device also includes a bus for connecting the memory 504 and the processor 503.

[0209] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the video stream encoding processing method performed by the video stream encoding processing device described above.

[0210] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0211] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0212] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method of video stream encoding processing, characterized by, The method comprises: determining an image sequence to be encoded and device parameters of a user equipment; determining initial encoding parameters corresponding to multiple quality levels according to the image sequence; dynamically adjusting the multiple initial encoding parameters according to the device parameters to obtain multiple target encoding parameters; encoding the image sequence according to the multiple target encoding parameters to obtain video streams corresponding to the multiple quality levels.

2. The method of claim 1, wherein, The determining of the initial encoding parameters corresponding to the multiple quality levels according to the image sequence comprises: analyzing the image sequence to obtain an analysis result, the analysis result being used to indicate scene information, complexity and texture details of the image sequence; determining encoding strategies corresponding to the multiple quality levels according to the scene information, complexity and texture details of the image sequence; determining the initial encoding parameters corresponding to each quality level based on the multiple encoding strategies, wherein the encoding parameters are used to indicate corresponding quantization parameters, frame rate control parameters and setting parameters of key frame intervals.

3. The method of claim 1, wherein, The device parameters comprise device performance and real-time network status, and the dynamically adjusting of the multiple initial encoding parameters according to the device parameters to obtain the multiple target encoding parameters comprises: determining an encoding parameter range of the user equipment according to the device performance and the real-time network status; judging whether any initial encoding parameter in the multiple initial encoding parameters is within the encoding parameter range; in a case where the initial encoding parameter is not within the encoding parameter range, adjusting the initial encoding parameter to be within the encoding parameter range to obtain a target encoding parameter; in a case where the initial encoding parameter is within the encoding parameter range, determining the initial encoding parameter as the target encoding parameter.

4. The method of claim 1, wherein, The encoding of the image sequence according to the multiple target encoding parameters to obtain the video streams corresponding to the multiple quality levels comprises: parallel processing of the video streams corresponding to the multiple quality levels by multiple encoders based on the multiple target encoding parameters; in the process of parallel processing by the multiple encoders, adjusting real-time code rates of the multiple encoders by using a code rate control algorithm.

5. The method of claim 3, wherein, After the encoding of the image sequence according to the multiple target encoding parameters to obtain the video streams corresponding to the multiple quality levels, the method further comprises: determining a corresponding transmission strategy according to the real-time network status, wherein different real-time network statuses correspond to different transmission strategies; determining a target transmission video stream from the video streams corresponding to the multiple quality levels according to the transmission strategy, and transmitting the target transmission video stream to the user equipment.

6. The method of claim 5, wherein, The method further comprises: in a case where the real-time network status changes, determining a new transmission strategy according to a new real-time network status; determining a target transmission video stream from the video streams corresponding to the multiple quality levels according to the new transmission strategy, and transmitting the target transmission video stream to the user equipment.

7. The method of claim 5, wherein, After the transmission of the target transmission video stream to the user equipment, the method further comprises: monitoring whether a transmission error occurs in the target transmission video stream, and determining error data in a case that a transmission error occurs in the target transmission video stream; adopting an error recovery mechanism to transmit the transmission video stream corresponding to the error data to the user equipment according to the error data.

8. A video stream encoding processing apparatus characterized by comprising: comprising: a determining module configured to determine an image sequence to be encoded and a device parameter of a user equipment; the determining module is further configured to determine initial encoding parameters corresponding to a plurality of quality levels according to the image sequence; an adjusting module configured to dynamically adjust a plurality of initial encoding parameters according to the device parameter to obtain a plurality of target encoding parameters; an encoding module configured to encode the image sequence according to the plurality of target encoding parameters to obtain video streams corresponding to the plurality of quality levels.

9. A video stream encoding processing apparatus characterized by comprising: comprising: a memory; a processor; wherein the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the video stream encoding processing method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by the processor to implement the video stream encoding processing method according to any one of claims 1-7.

Images