Video playback method and device, electronic equipment and storage medium

Abstract

This invention provides a video playback method, apparatus, electronic device, and storage medium, relating to the field of video processing technology. In this method, each main video frame of the target video corresponds to a preset number of thumbnail frames. Each thumbnail frame is encoded to obtain a sub-stream frame. This achieves the effect of consuming only the storage space required for a single resolution sub-stream while outputting sub-streams of multiple resolutions as playback streams. This not only saves server storage space, but also eliminates the need for users to resend multiple resolution streams by simply adjusting the number of sent sub-stream frames when switching resolutions, thus ensuring smooth client-side image presentation and reducing the possibility of stuttering or black screens. Furthermore, it avoids the repeated consumption of device performance due to multiple re-encoding during streaming, shortening client-side image output time and improving user experience.

Description

Technical Field

[0001] This invention relates to the field of video processing technology, and in particular to a video playback method, apparatus, electronic device, and storage medium. Background Technology

[0002] In recent years, mobile monitoring has been widely used in various scenarios in the field of video surveillance.

[0003] During video playback, users have varying needs for playback clarity in different scenarios. One approach is to store streams of different resolutions to meet these varying requirements. However, storing multiple resolution streams consumes significant storage space. Furthermore, switching resolutions may cause stuttering or blackouts, negatively impacting the user experience. Another approach is to store only high-resolution video and then re-encode it into low-resolution video before sending it to the client. This method requires re-encoding every time a stream is sent, leading to redundant resource consumption, longer image processing times, and a poor user experience.

[0004] Therefore, there is an urgent need to provide a video playback method. Summary of the Invention

[0005] This invention provides a video playback method, apparatus, electronic device, and storage medium to address the deficiencies in related technologies.

[0006] This invention provides a video playback method applied to a server, wherein the server stores a target video, including: The system receives a streaming start signaling from the client for the target video and sequentially sends a preset number of sub-stream frames of the first intra-coded frame of the target video to the client. Determine the current resolution level, and based on the mapping relationship between the resolution level and the sub-stream frames, determine the remaining sub-stream frames corresponding to the current resolution level; Send the current resolution level and the remaining sub-stream frames corresponding to the current resolution level to the client; Each main video frame of the target video corresponds to the preset number of thumbnail frames, and each thumbnail frame is encoded to obtain a sub-stream frame.

[0007] According to a video playback method provided by the present invention, sending the current resolution level and the remaining sub-stream frames corresponding to the current resolution level to the client specifically includes: The client is sent the current resolution level, the remaining sub-stream frames corresponding to the current resolution level, and the mapping relationship.

[0008] According to a video playback method provided by the present invention, determining the current resolution level includes: The current resolution level is determined based on the number of sub-stream frames successfully sent to the client within each preset time period.

[0009] According to a video playback method provided by the present invention, the step of determining the remaining sub-stream frames corresponding to the current resolution level based on the mapping relationship between resolution level and sub-stream frames includes the following steps: Based on each sub-stream frame corresponding to each mainstream video frame, determine the sub-stream sequence number for different resolution levels and the sub-stream frame sequence number associated with each sub-stream sequence number. The mapping relationship is established based on the resolution level corresponding to each sub-stream sequence number and the associated sub-stream frame sequence number.

[0010] According to a video playback method provided by the present invention, the step of receiving a streaming signaling message from a client for the target video includes: The preset quantity is sent to the client, which displays the preset quantity, receives user feedback, and sends the feedback to the server. The system receives the feedback information and applies the mapping relationship to send the remaining sub-stream frames of the resolution level corresponding to the feedback information to the client.

[0011] The present invention also provides a video playback method, applied to a client, comprising: Send a streaming start signal to the server for the target video, and receive each sub-stream frame of the target video and the current resolution level from the server. Based on the current resolution level, each sub-stream frame is rendered into an image.

[0012] According to a video playback method provided by the present invention, the step of rendering images of each sub-stream frame based on the current resolution level includes: Decode each sub-stream frame to obtain a thumbnail frame of each sub-stream frame; Based on the current resolution level, determine whether there are any missing target frames when the thumbnail frames corresponding to each sub-stream are overlaid. If the target image frame exists, the pixel value of each pixel in the target image frame is determined based on the pixel value of each pixel in the thumbnail frame corresponding to each sub-bitstream. The thumbnail frames corresponding to each sub-stream and the target frame are overlaid and rendered.

[0013] According to a video playback method provided by the present invention, the step of rendering images of each sub-stream frame based on the current resolution level further includes: Receive the mapping relationship between resolution level and sub-stream frame sent by the server; Based on the mapping relationship and the current resolution level, each sub-stream frame is rendered into an image.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the video playback method as described above.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the video playback method as described above.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the video playback method as described above.

[0017] The video playback method, apparatus, electronic device, and storage medium provided by this invention involve a method where each main video frame of the target video corresponds to a preset number of thumbnail frames. Each thumbnail frame is encoded to obtain a sub-stream frame. This achieves the effect of consuming only the storage space required for a sub-stream of one resolution level while outputting sub-streams of multiple resolution levels as playback streams. This not only saves server storage space, but also eliminates the need for users to resend multiple resolution streams by simply adjusting the number of sub-stream frames sent when switching resolutions, thus ensuring smooth client-side image presentation and reducing the possibility of stuttering or black screens. Furthermore, it avoids the repeated consumption of device performance caused by multiple re-encoding during streaming, shortening the client's image output time and improving the user experience. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is one of the flowcharts of the video playback method provided by the present invention.

[0020] Figure 2This is a schematic diagram showing the resolution of each thumbnail frame when the preset number is 2 in the video playback method provided by the present invention.

[0021] Figure 3 This is a schematic diagram showing the resolution of each thumbnail frame when the preset number is 4 in the video playback method provided by the present invention.

[0022] Figure 4 This is the second flowchart of the video playback method provided by the present invention.

[0023] Figure 5 This is a partial schematic diagram showing the resolution of each thumbnail frame when the preset number is 4 in the video playback method provided by the present invention.

[0024] Figure 6 This is one of the structural schematic diagrams of the video playback device provided by the present invention.

[0025] Figure 7 This is the second structural schematic diagram of the video playback device provided by the present invention.

[0026] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0028] Existing technologies, limited by video storage methods, can degrade the user experience in video playback scenarios. Therefore, this invention provides a video playback method.

[0029] Figure 1 This is a flowchart illustrating a video playback method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, this method is applied to a server that stores the target video, including: S11, Receive the starting signaling from the client for the target video, and sequentially send a preset number of sub-stream frames of the first intra-frame coded frame of the mainstream of the target video to the client; S12, determine the current resolution level, and based on the mapping relationship between the resolution level and the sub-stream frame, determine the remaining sub-stream frame corresponding to the current resolution level; S13, send the current resolution level and the remaining sub-stream frames corresponding to the current resolution level to the client; Each main video frame of the target video corresponds to the preset number of thumbnail frames, and each thumbnail frame is encoded to obtain a sub-stream frame.

[0030] Specifically, the video playback method provided in this embodiment of the invention uses a video playback device as its execution entity. This device can be divided into a storage device and a sending device, configured within a server. The server can store the target video. In response to a client's initiation signal for the target video, the server sends the playback stream of the target video to the client, enabling the client to play the playback stream. The storage device stores the target video, and the sending device sends the playback stream of the target video to the client.

[0031] First, step S11 is executed, receiving a streaming start signal from the client for the target video. This streaming start signal can be initiated by the user launching the client and opening the playback screen generation. This streaming start signal can be used to instruct the server to start the generation and transmission process of the playback stream.

[0032] Understandably, within the server, the storage device can divide each mainstream video frame (i.e., each mainstream video frame) of the target video into a preset number of N thumbnail frames for storage, and the mainstream image can be obtained by overlaying and rendering each thumbnail frame after decoding. Therefore, the server can support the transmission of a preset number of sub-streams with different resolution levels through the transmission device.

[0033] When N is 2, the resolution of each thumbnail frame is as follows: Figure 2 As shown, when N is 4, the resolution of each thumbnail frame is as follows: Figure 3 As shown. Each color of pixels is combined to form a thumbnail frame, and all colors of pixels are combined to form a main video frame.

[0034] When storing N thumbnail frames for each main video frame, the storage device can encode each thumbnail frame using encoding algorithms such as H.264 or H.265 to obtain a sub-stream frame. Therefore, each main video frame of the target video corresponds to N sub-stream frames, and each intra-coded picture (I-frame) and each predicted frame (P-frame) obtained after encoding the target video also includes N sub-stream frames.

[0035] In response to the stream initiation signaling, the server can sequentially send the first I-frame of the target video's main stream, along with N sub-stream frames, to the client via a sending device. Upon receiving each sub-stream frame, the client can immediately render each sub-stream frame.

[0036] Then, step S12 is executed. The server can determine the current resolution level through the sending device. The current resolution level can be the initial resolution level given by default by the sending device, or it can be determined by the sending device based on the network quality during the interaction with the client during the transmission of N sub-stream frames in the first I-frame. No specific limitation is made here.

[0037] Understandably, the current resolution level is used to represent the resolution level of the sub-stream that needs to be sent to the client. Since the server can support the transmission of streams with a total of N resolution levels, the current resolution level can be any of the N resolution levels, and can default to the lowest level, i.e., level 1.

[0038] During the storage process of a server's storage device, a resolution level can be associated one-to-one with a sub-stream, and each sub-stream can be composed of one or more sub-stream frames. Therefore, the mapping relationship between resolution levels and sub-stream frames can be determined. This mapping relationship can be a correspondence between a resolution level and the sub-stream frame number that identifies each sub-stream frame; one resolution level can correspond to one or more sub-stream frames, and there is a mapping relationship between one or more sub-stream frame numbers.

[0039] Using this mapping relationship, the remaining sub-stream frames corresponding to the current resolution level can be determined. These remaining sub-stream frames can be sub-stream frames that were not sent to the client in the first I-frame, or they can be sub-stream frames that need to be transmitted to the client in other I-frames or P-frames. No specific limitation is made here.

[0040] Finally, step S13 is executed, in which the determined current resolution level and the remaining sub-stream frames corresponding to the current resolution level are sent to the client in sequence using the sending device.

[0041] After receiving the current resolution level and the remaining sub-stream frames corresponding to the current resolution level, the client can use the current resolution level to determine the target number of sub-stream frames in the sub-stream sent by the server, and then sequentially overlay and render the target number of sub-stream frames in the remaining sub-stream frames corresponding to the current resolution level and play the image.

[0042] The video playback method provided in this embodiment of the invention is applied to a server. The server stores a target video. First, it receives a streaming start signal from the client for the target video and sequentially sends a preset number of sub-stream frames of the first intra-coded frame of the target video's main stream to the client. Then, it determines the current resolution level and uses the mapping relationship between the resolution level and the sub-stream frames to determine the remaining sub-stream frames corresponding to the current resolution level. Finally, it sends the current resolution level and the remaining sub-stream frames corresponding to the current resolution level to the client. In this method, each main video frame of the target video corresponds to a preset number of thumbnail frames. Each thumbnail frame is encoded to obtain a sub-stream frame, which can achieve the effect of consuming only the storage space required for a sub-stream of one resolution level while outputting sub-streams of multiple resolution levels as playback streams. This not only saves server storage space, but also allows users to switch resolutions simply by adjusting the number of sent sub-stream frames, eliminating the need to resend multiple resolution streams, thus achieving smooth presentation of the client's screen and reducing the possibility of stuttering or black screens. This method also avoids the repeated consumption of device performance caused by multiple re-encodings during transmission, shortens the client's image output time, and improves the user experience. Moreover, since the client can output an image based on the first I-frame in this method, even if packet loss occurs at the end of a preset number of thumbnail frames of a mainstream video frame during transmission, there is no need to retransmit. It is only necessary to send the next I-frame and subsequent P-frames of appropriate resolution, which can improve video playback efficiency.

[0043] Based on the above embodiments, sending the current resolution level and the remaining sub-stream frames corresponding to the current resolution level to the client specifically includes: The client is sent the current resolution level, the remaining sub-stream frames corresponding to the current resolution level, and the mapping relationship.

[0044] Specifically, when sending the current resolution level and the remaining sub-stream frames corresponding to the current resolution level to the client, in order to avoid the server sending order and the client receiving order being different due to network delays or other reasons, which would lead to overlay rendering errors on the client, the mapping relationship between the resolution level and the sub-stream frames can also be sent to the client. The client can then determine each sub-stream frame of the current resolution level based on this mapping relationship, and then overlay and render an accurate image for playback.

[0045] When users watch playback on mobile devices, if the network quality on the client side is poor, high-resolution image rendering will be slow because packet loss may occur during transmission. Once a packet is lost, the previously sent data is useless and needs to be retransmitted. Moreover, if users want faster image rendering, they need to manually switch to low resolution. Even if there is a strategy for automatically switching to low resolution, current technology still needs to go through a time lag in sensing the network before reducing the resolution, resulting in a relatively long image rendering time and unsatisfactory user experience.

[0046] Based on this, and building upon the above embodiments, determining the current resolution level includes: The current resolution level is determined based on the number of sub-stream frames successfully sent to the client within each preset time period.

[0047] Specifically, the server sends a sub-stream frame to the client. If the transmission is successful, the server will receive a reception feedback from the client indicating that the sub-stream frame has been successfully received. The server considers itself to have successfully sent the sub-stream frame to the client upon receiving the client's reception feedback.

[0048] Therefore, the number of sub-stream frames successfully sent to the client within each preset time period can indicate the network quality of the client during that preset time period. If the number of sub-stream frames successfully sent to the client within a preset time period is small, it indicates that the network quality of the client during that preset time period is poor. Otherwise, if the number of sub-stream frames successfully sent to the client within a preset time period is large, it indicates that the network quality of the client during that preset time period is good.

[0049] Furthermore, the current resolution level can be determined by the number of sub-stream frames successfully sent to the client within each preset time period. The duration M of this preset time period can be set as needed, and can be on the order of milliseconds.

[0050] Here, during the transmission of each sub-stream frame in the first I-frame, starting from the transmission of the first sub-stream frame, the number of sub-stream frames successfully transmitted to the client within each preset time period is determined in real time.

[0051] When a sub-stream frame of the first I-frame is successfully sent to the client, the client can play the image. If, after M milliseconds, subsequent sub-stream frames have not arrived and no feedback on the reception of sub-stream frames is sent to the server, it indicates that the network quality of the client is poor during this preset time period. The server's sending device then determines that only one sub-stream frame was successfully sent to the client within the preset time period, and therefore determines that the current resolution level is 1. Subsequent streams only need to send 1 / N of the image. The client then abandons using the complete high-definition image and only plays 1 / N of the original resolution image.

[0052] When a sub-stream frame of the first I-frame is successfully sent to the client, the client can first play the picture. Within M milliseconds after the picture appears, if the subsequent second sub-stream frame arrives in time, it indicates that the network quality of the client within the preset time period is relatively good. Then the client continues to wait for the subsequent sub-stream frames and sends a reception feedback to the server after successfully receiving the sub-stream frames. If the sending device of the server determines that X (1 < X < N) sub-stream frames are successfully sent to the client within the preset time period, it determines that the current clarity level is X, and only X / N of the pictures need to be sent in the subsequent stream sending. The client then plays the subsequent stream using the clarity of level X.

[0053] By determining the current clarity level based on the number of sub-stream frames successfully sent to the client within each preset time period during the sending process of a preset number of sub-stream frames of the first I-frame, the detection of the network quality can be achieved during the picture appearance of the first frame, and the effect of quickly selecting an appropriate clarity level can be achieved.

[0054] On this basis, after determining the current clarity level, stop sending the remaining sub-stream frames in the first I-frame to the client, and send sub-stream frames according to the current clarity level during the picture appearance of the first frame, achieving the dual effects of quickly presenting the first frame and detecting the network quality.

[0055] It can be understood that the number of sub-stream frames successfully sent to the client within each preset time period can be judged in real time. Therefore, during the sending process of each sub-stream frame in non-first I-frames, the number of sub-stream frames successfully sent to the client within each preset time period can also be judged.

[0056] Generally, if the sending device of the server determines that the number a of sub-stream frames successfully sent to the client in a subsequent preset time period is more than the number b of sub-stream frames successfully sent to the client in a previous preset time period, that is, a > b, it indicates that the network quality of the client changes from poor to good, and the current clarity level changes from b to a. The server needs to change the picture sending from b / N to a / N.

[0057] If the sending device of the server determines that the number a of sub-stream frames successfully sent to the client in a subsequent preset time period is less than the number b of sub-stream frames successfully sent to the client in a previous preset time period, that is, a < b, it indicates that the network quality of the client changes from good to poor, and the current clarity level changes from b to a. The server needs to change the picture sending from b / N to a / N.

[0058] In this embodiment of the invention, the server's sending device can adaptively determine the current resolution level by the number of sub-stream frames successfully sent to the client within each preset time period. It can quickly select a suitable resolution level based on the real-time network quality of the client, thereby balancing video playback efficiency and image clarity, achieving the goal of rapid image output for the client, and avoiding a decline in user experience due to the time-consuming image output process.

[0059] Based on the above embodiments, the step of determining the remaining sub-stream frames corresponding to the current resolution level based on the mapping relationship between resolution level and sub-stream frames includes the following: Based on each sub-stream frame corresponding to each mainstream video frame, determine the sub-stream sequence number for different resolution levels and the sub-stream frame sequence number associated with each sub-stream sequence number. The mapping relationship is established based on the resolution level corresponding to each sub-stream sequence number and the associated sub-stream frame sequence number.

[0060] Specifically, before applying the mapping relationship between resolution levels and sub-stream frames, the server's storage device needs to determine the mapping relationship when storing the target video.

[0061] When determining the mapping relationship, a preset number of sub-stream frames obtained by segmenting each main video frame of the target video can be used to determine the sub-stream sequence number of different resolution levels and the sub-stream frame sequence number associated with each sub-stream sequence number.

[0062] Each resolution level can correspond to a sub-stream sequence number, which represents the sub-stream at that resolution level. Each sub-stream sequence number can be associated with one or more sub-stream frames, the number of which is the same as the corresponding resolution level. For example, if the sub-stream sequence number for resolution level d is d1, then the resolution level of sub-stream e identified by sub-stream sequence number d1 is d. Furthermore, the sub-stream frames associated with sub-stream sequence number d1 include d frames, and these d frames constitute sub-stream e. Each sub-stream frame is identified by its sub-stream frame sequence number.

[0063] Subsequently, a mapping relationship can be established using the resolution level corresponding to each sub-stream sequence number and the associated sub-stream frame sequence number. That is, the resolution level corresponding to the same sub-stream sequence number is bound to the sub-stream frame sequence number to obtain the mapping relationship.

[0064] In this embodiment of the invention, by establishing a mapping relationship between the resolution level corresponding to the same sub-stream sequence number and the sub-stream frame sequence number, sub-stream frames can be sent to the client in the order of the sub-stream frame sequence numbers corresponding to the current resolution level. This allows the client to accurately overlay and render the sub-stream frames for playback. Alternatively, when the mapping relationship is sent to the client, the client can quickly determine the sub-stream frame sequence number corresponding to the current resolution level based on the mapping relationship, and then overlay and render the sub-stream frames in the order of the sub-stream frame sequence numbers for playback.

[0065] Based on the above embodiments, the receiving client then sends a streaming start signaling for the target video, followed by: The preset quantity is sent to the client, which displays the preset quantity, receives user feedback, and sends the feedback to the server. The system receives the feedback information and applies the mapping relationship to send the remaining sub-stream frames of the resolution level corresponding to the feedback information to the client.

[0066] Specifically, after receiving the streaming start signal from the client for the target video, a preset number N can be sent to the client. This preset number N is the number of resolution levels that the server can support, and the current resolution level can be set to 1 by default.

[0067] After receiving the preset quantity N, the client can display the preset quantity N so that the user can adjust the resolution level between 1 and N according to their own needs, and then feed the adjusted resolution level back to the server through the client.

[0068] The server then receives feedback from the user via the client, including the adjusted resolution level. Subsequently, the server's storage device, based on this feedback and applying a mapping relationship, sends the remaining sub-stream frames corresponding to the resolution level to the client over the same transmission link, enabling a smooth display of image clarity on the client side.

[0069] After receiving the remaining sub-stream frames, the client can overlay and render the remaining sub-stream frames according to the resolution level corresponding to the feedback information to output the image for playback.

[0070] like Figure 4 As shown, based on the above embodiments, this embodiment of the invention also provides a video playback method, which is applied to a client and includes: S21, send a streaming start signaling message for the target video to the server, and receive each sub-stream frame of the target video and the current resolution level sent by the server. S22, based on the current resolution level, render each sub-stream frame into an image.

[0071] Specifically, the video playback method provided in this embodiment of the invention is executed by a video playback device, which can be a playback device configured in a client. The client can send a streaming start signal to the server for the target video, receive the playback bitstream of the target video sent by the server, decode the playback bitstream, and output the image for playback.

[0072] First, step S21 is executed, sending a streaming start signaling message for the target video to the server, and receiving each sub-stream frame of the target video and the current resolution level as determined and sent by the server through the video playback method provided in the above embodiments.

[0073] Then, step S22 is executed. The number of sub-stream frames in the sub-stream sent by the server can be determined by the current resolution level. Based on the received sub-stream frames, the sub-stream frames corresponding to the current resolution level are overlaid, rendered, and played out.

[0074] The video playback method provided in this embodiment of the invention is applied to a client. First, it sends a streaming start signal to the server for the target video and receives each sub-stream frame of the target video and the current resolution level from the server. Then, based on the current resolution level, it renders each sub-stream frame into an image. This method allows the client to quickly render images at the current resolution level by having the server send the required sub-stream frames to the client, thus improving the user experience. Furthermore, this method can save storage space occupied by the target video on the server.

[0075] Based on the above embodiments, the step of rendering each sub-stream frame based on the current resolution level includes: Decode each sub-stream frame to obtain a thumbnail frame of each sub-stream frame; Based on the current resolution level, determine whether there are any missing target frames when the thumbnail frames corresponding to each sub-stream are overlaid. If the target image frame exists, the pixel value of each pixel in the target image frame is determined based on the pixel value of each pixel in the thumbnail frame corresponding to each sub-bitstream. The thumbnail frames corresponding to each sub-stream and the target frame are overlaid and rendered.

[0076] Specifically, when rendering each sub-stream frame, each received sub-stream frame can be decoded first to obtain a thumbnail frame of each sub-stream frame.

[0077] Then, using the current resolution level, i.e., the number of sub-stream frames corresponding to each sub-stream, it is determined whether there are any missing target frames when the thumbnail frames corresponding to each sub-stream are overlaid. For example... Figure 5 As shown, N=4, meaning that each sub-stream has four different colors of pixels. If the current resolution level is 3, then the number of sub-stream frames corresponding to each sub-stream is 3, and each sub-stream frame corresponds to a pixel of one color.

[0078] Therefore, the server only transmits images at 3 / 4 resolution of the original image, meaning it only needs to transmit the sub-stream frames corresponding to the three different colors of pixels in each sub-stream. If the server only transmits... Figure 5 The client determines that when the thumbnail frames corresponding to the pixels of the three different colors "black, red, and gray" are superimposed, there are missing target frames.

[0079] The target image frame is a thumbnail frame composed of green pixels. In this embodiment of the invention, bilinear interpolation can be used, that is, the pixel value of each pixel in the target image frame is estimated by calculating the weighted average of the four nearest neighbor pixels of each green pixel. Subsequently, the thumbnail frames corresponding to each sub-stream and the target image frame are superimposed and rendered to obtain an image with 3 / 4 the resolution of the original image.

[0080] In this embodiment of the invention, on the client side, the pixel value of each pixel in the target image frame can be determined by pixel compensation, thereby supporting flexible switching of a preset number of resolution levels.

[0081] If there are no missing target frames when the thumbnail frames corresponding to each sub-stream are overlaid, for example, if the current resolution level is 1, that is, the number of sub-stream frames corresponding to each sub-stream is 1, and each sub-stream corresponds to only one color pixel.

[0082] Therefore, the server only transmits an image with 1 / 4 the resolution of the original image. That is, it only needs to transmit the sub-stream frames corresponding to the pixels of one color in each sub-stream. At this time, the thumbnail frames are proportional to the original image, and there are no missing target frames. The client determines that the thumbnail frames corresponding to this sub-stream do not need to be superimposed and can be directly rendered.

[0083] If the current resolution level is 2, meaning each sub-stream corresponds to 2 sub-stream frames, and each sub-stream corresponds to pixels of two colors, then the server only transmits an image at 2 / 4 resolution of the original image. This means it only needs to transmit the sub-stream frames corresponding to the two colors of each sub-stream. Since the superimposed thumbnail frames of this sub-stream are only proportional to the original image in width or height, the client determines that there are no missing target frames when the thumbnail frames of this sub-stream are superimposed, and can directly render the image.

[0084] Based on the above embodiments, the step of rendering each sub-stream frame based on the current resolution level further includes: Receive the mapping relationship between resolution level and sub-stream frame sent by the server; Based on the mapping relationship and the current resolution level, each sub-stream frame is rendered into an image.

[0085] Specifically, in this embodiment of the invention, the client can also receive the mapping relationship between the resolution level and the sub-stream frames sent by the server. Then, the client can use the mapping relationship to determine the sub-stream frame number corresponding to the current resolution level, and render the sub-stream frames corresponding to the current resolution level according to the order of the sub-stream frame numbers. This can avoid the problem of the server sending order and the client receiving order being different due to network latency and other reasons, which would lead to the client overlay rendering error. This can ensure the accuracy of the output image.

[0086] like Figure 6 As shown, based on the above embodiments, this embodiment of the invention provides a video playback device applied to a server, wherein the server stores a target video, including: The first sending module 61 is used to receive the starting signaling of the target video from the client and send a preset number of sub-stream frames of the first intra-frame encoded frame of the target video to the client in sequence. The resolution level determination module 62 is used to determine the current resolution level and, based on the mapping relationship between the resolution level and the sub-stream frame, determine the remaining sub-stream frame corresponding to the current resolution level. The second sending module 63 is used to send the current resolution level and the remaining sub-stream frames corresponding to the current resolution level to the client. Each main video frame of the target video corresponds to the preset number of thumbnail frames, and each thumbnail frame is encoded to obtain a sub-stream frame.

[0087] Based on the above embodiments, the video playback device provided in this embodiment of the invention, the second sending module, is specifically used for: The client is sent the current resolution level, the remaining sub-stream frames corresponding to the current resolution level, and the mapping relationship.

[0088] Based on the above embodiments, the video playback device provided in this embodiment of the invention, wherein the gear position determination module is specifically used for: The current resolution level is determined based on the number of sub-stream frames successfully sent to the client within each preset time period.

[0089] Based on the above embodiments, the video playback device provided in this embodiment of the invention further includes a relationship establishment module, used for: Based on each sub-stream frame corresponding to each mainstream video frame, determine the sub-stream sequence number for different resolution levels and the sub-stream frame sequence number associated with each sub-stream sequence number. The mapping relationship is established based on the resolution level corresponding to each sub-stream sequence number and the associated sub-stream frame sequence number.

[0090] Based on the above embodiments, the video playback device provided in this embodiment of the invention further includes a third sending module, used for: The preset quantity is sent to the client, which displays the preset quantity, receives user feedback, and sends the feedback to the server. The system receives the feedback information and applies the mapping relationship to send the remaining sub-stream frames of the resolution level corresponding to the feedback information to the client.

[0091] Specifically, the functions of each module in the video playback device provided in this embodiment of the invention correspond one-to-one with the operation flow of each step in the above-described method embodiment with the video playback device configured in the server as the execution subject, and the achieved effect is also the same. For details, please refer to the above embodiments, and this will not be repeated in this embodiment of the invention.

[0092] like Figure 7 As shown, based on the above embodiments, this embodiment of the invention also provides a video playback device applied to a client, wherein the server stores a target video, including: The receiving module 71 is used to send a streaming start signaling for the target video to the server, and to receive each sub-stream frame of the target video and the current resolution level sent by the server. The rendering module 72 is used to render images of each sub-stream frame based on the current resolution level.

[0093] Based on the above embodiments, the video playback device provided in this embodiment of the invention, wherein the rendering module is specifically used for: Decode each sub-stream frame to obtain a thumbnail frame of each sub-stream frame; Based on the current resolution level, determine whether there are any missing target frames when the thumbnail frames corresponding to each sub-stream are overlaid. If the target image frame exists, the pixel value of each pixel in the target image frame is determined based on the pixel value of each pixel in the thumbnail frame corresponding to each sub-bitstream. The thumbnail frames corresponding to each sub-stream and the target frame are overlaid and rendered.

[0094] Based on the above embodiments, the video playback device provided in this embodiment of the invention, wherein the rendering module is further specifically used for: Receive the mapping relationship between resolution level and sub-stream frame sent by the server; Based on the mapping relationship and the current resolution level, each sub-stream frame is rendered into an image.

[0095] Specifically, the functions of each module in the video playback device provided in this embodiment of the invention correspond one-to-one with the operation flow of each step in the above-described method embodiment with the video playback device configured in the client as the execution subject, and the achieved effect is also the same. For details, please refer to the above embodiments, and this will not be repeated in this embodiment of the invention.

[0096] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute the video playback methods provided in the above embodiments.

[0097] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to related technologies, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0098] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the video playback method provided in the above embodiments.

[0099] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the video playback methods provided in the above embodiments.

[0100] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0101] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video playback method, characterized by, Applied to a server, the server storing the target video, including: The system receives a streaming start signaling from the client for the target video and sequentially sends a preset number of sub-stream frames of the first intra-coded frame of the target video to the client. Determine the current resolution level, and based on the mapping relationship between the resolution level and the sub-stream frames, determine the remaining sub-stream frames corresponding to the current resolution level; Send the current resolution level and the remaining sub-stream frames corresponding to the current resolution level to the client; Each main video frame of the target video corresponds to the preset number of thumbnail frames, and each thumbnail frame is encoded to obtain a sub-stream frame.

2. The video playback method of claim 1, wherein, Sending the current resolution level and the remaining sub-stream frames corresponding to the current resolution level to the client specifically includes: The client is sent the current resolution level, the remaining sub-stream frames corresponding to the current resolution level, and the mapping relationship.

3. The video playback method of claim 1, wherein, Determining the current resolution level includes: The current resolution level is determined based on the number of sub-stream frames successfully sent to the client within each preset time period.

4. The video playback method of any of claims 1-3, wherein, The step of determining the remaining sub-stream frames corresponding to the current resolution level based on the mapping relationship between resolution level and sub-stream frames includes: Based on each sub-stream frame corresponding to each mainstream video frame, determine the sub-stream sequence number for different resolution levels and the sub-stream frame sequence number associated with each sub-stream sequence number. The mapping relationship is established based on the resolution level corresponding to each sub-stream sequence number and the associated sub-stream frame sequence number.

5. The video playback method of any of claims 1-3, wherein, The receiving client then initiates the streaming signaling for the target video, followed by: The preset quantity is sent to the client, which displays the preset quantity, receives user feedback, and sends the feedback to the server. The system receives the feedback information and applies the mapping relationship to send the remaining sub-stream frames of the resolution level corresponding to the feedback information to the client.

6. A video playback method characterized by, Applied to the client side, including: Send a streaming start signal to the server for the target video, and receive each sub-stream frame of the target video and the current resolution level from the server. Based on the current resolution level, each sub-stream frame is rendered into an image.

7. The video playback method of claim 6, wherein, The step of rendering images from each sub-stream frame based on the current resolution level includes: Decode each sub-stream frame to obtain a thumbnail frame of each sub-stream frame; Based on the current resolution level, determine whether there are any missing target frames when the thumbnail frames corresponding to each sub-stream are overlaid. If the target image frame exists, the pixel value of each pixel in the target image frame is determined based on the pixel value of each pixel in the thumbnail frame corresponding to each sub-bitstream. The thumbnail frames corresponding to each sub-stream and the target frame are overlaid and rendered.

8. The video playback method of claim 6, wherein, The step of rendering images from each sub-stream frame based on the current resolution level also includes: Receive the mapping relationship between resolution level and sub-stream frame sent by the server; Based on the mapping relationship and the current resolution level, each sub-stream frame is rendered into an image.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the video playback method as described in any one of claims 1-8.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the video playback method as described in any one of claims 1-8.

Images