Method for sending video stream, related device and computer program product

Abstract

The present disclosure provides a method, device, electronic device, computer-readable storage medium and computer program product for sending video streams, which relate to cloud computing technology fields such as data processing, cloud platform, and smart cloud, and can be applied to scenarios such as cloud gaming. A specific implementation of the method includes: after determining the target video stream to be sent, extracting image frames in the target video stream one by one according to a preset interval period, generating a first corrected video stream, encoding the first corrected video stream, generating a coded data stream, and finally sending the coded data stream to the target terminal device. This implementation provides a method for sending a video stream, which can correct the target video stream to be sent during the sending process to avoid frame skipping in the target video stream due to problems such as encoding and transmission delay, and can improve the quality of the sent target video stream.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, specifically to the field of cloud computing technology such as data processing, cloud platform, and intelligent cloud, and especially to a method, device, electronic device, computer-readable storage medium, and computer program product for sending video streams. Background Art

[0002] Cloud gaming is a gaming method based on cloud computing. In this mode, all games are run on cloud servers, and the rendered game images are compressed and transmitted to users over the network. In this case, the user's terminal device only needs basic video decompression capabilities to achieve the same gaming experience as high-end processors and graphics cards, without the need for high-end processors and graphics cards.

[0003] The key to the cloud gaming user experience lies in the smoothness of the graphics, which affects the user's operational experience. The smoothness of the graphics depends on the client network environment and the stability of the cloud device's encoded video output data. Summary of the Invention

[0004] The embodiments of the present disclosure provide a method, apparatus, electronic device, computer-readable storage medium, and computer program product for delivering a video stream.

[0005] In the first aspect, an embodiment of the present disclosure proposes a method for sending a video stream, including: determining a target video stream to be sent; extracting image frames in the target video stream one by one according to a preset interval period to generate a first corrected video stream; encoding the first corrected video stream to generate a coded data stream; and sending the coded data stream to a target terminal device.

[0006] In the second aspect, an embodiment of the present disclosure proposes a device for sending a video stream, comprising: a video stream acquisition unit, configured to determine a target video stream to be sent; a first video stream correction unit, configured to extract image frames in the target video stream one by one according to a preset interval period to generate a first corrected video stream; a coded data stream generation unit, configured to encode the first corrected video stream to generate a coded data stream; and a coded data stream sending unit, configured to send the coded data stream to a target terminal device.

[0007] In a third aspect, an embodiment of the present disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that when the at least one processor executes the method for sending a video stream as described in any implementation method in the first aspect, it can be implemented.

[0008] In a fourth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, which are used to enable a computer to implement the method for sending a video stream as described in any implementation method of the first aspect when executed.

[0009] In a fifth aspect, an embodiment of the present disclosure provides a computer program product including a computer program, which, when executed by a processor, can implement the method for sending a video stream as described in any implementation manner in the first aspect.

[0010] The method, device, electronic device, computer-readable storage medium and computer program product for sending video streams provided by the embodiments of the present disclosure, after determining the target video stream to be sent, extract the image frames in the target video stream one by one according to a preset interval period, generate a first corrected video stream, encode the first corrected video stream, generate a coded data stream, and finally send the coded data stream to the target terminal device.

[0011] The present invention can correct the video stream to be sent during the video stream sending process by extracting image frames from the target video stream to be sent according to a preset interval period and constructing a corrected video stream, so as to avoid frame skipping in the target video stream due to problems such as encoding and transmission delay, and improve the quality of the sent target video stream.

[0012] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Other features, objects and advantages of the present disclosure will become more apparent from a reading of the detailed description of non-limiting embodiments made with reference to the following drawings:

[0014] Figure 1 is an exemplary system architecture in which the present disclosure may be applied;

[0015] Figure 2 A flowchart of a method for delivering a video stream provided by an embodiment of the present disclosure;

[0016] Figure 3 A flowchart of another method for delivering a video stream provided by an embodiment of the present disclosure;

[0017] Figure 4 A flowchart of a method for delivering a video stream in an application scenario provided by an embodiment of the present disclosure;

[0018] Figure 5 A structural block diagram of a device for sending a video stream provided by an embodiment of the present disclosure;

[0019] Figure 6 A schematic structural diagram of an electronic device suitable for executing a method for sending a video stream provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. It should be noted that the embodiments in the present disclosure and the features in the embodiments can be combined with each other unless there is a conflict.

[0021] In addition, in the technical solutions involved in this disclosure, the acquisition, storage, use, processing, transportation, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0022] Figure 1 An exemplary system architecture 100 is shown to which embodiments of the method, apparatus, electronic device, and computer-readable storage medium for delivering a video stream of the present disclosure may be applied.

[0023] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is a medium for providing communication links between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0024] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various applications for enabling information communication between the terminal devices 101, 102, and 103 and server 105 can be installed, such as cloud gaming applications, cloud video applications, and instant messaging applications.

[0025] Terminal devices 101, 102, 103 and server 105 can be either hardware or software. When terminal devices 101, 102, 103 are hardware, they can be various electronic devices with display screens, including but not limited to smartphones, tablet computers, laptop computers, and desktop computers. When terminal devices 101, 102, 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software or software modules, or as a single software or software module, and are not specifically limited here. When server 105 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software or software modules, or as a single software or software module, and are not specifically limited here.

[0026] The server 105 can provide various services through various built-in applications. Taking the cloud gaming application that can provide cloud gaming services as an example, the server 105 can achieve the following effects when running the cloud gaming application: first, the server 105 determines the target video stream to be sent; then, the server 105 extracts the image frames in the target video stream one by one according to a preset interval period to generate a first corrected video stream; next, the server 105 encodes the first corrected video stream to generate a coded data stream; finally, the server 105 sends the coded data stream to the target terminal devices 101, 102, and 103 through the network 104.

[0027] It should be pointed out that in addition to being pre-stored locally on the server 105 in various ways, the target video stream can also be stored in a non-local terminal device (such as terminal devices 101, 102, 103) or server. In this case, the server 105 can obtain the corresponding target video stream by sending a target video stream acquisition instruction to the non-local terminal device or server that stores the target video stream.

[0028] Since storing and encoding the target video stream requires a significant amount of computing resources and significant computing power, the methods for delivering video streams provided in the subsequent embodiments of this disclosure are generally performed by a server 105 with significant computing power and resources. Accordingly, the apparatus for delivering the video stream is generally also located within the server 105. However, it should also be noted that, if the terminal devices 101, 102, and 103 also possess sufficient computing power and resources, the terminal devices 101, 102, and 103 can also perform the aforementioned operations delegated to the server 105 through the cloud gaming application installed thereon, thereby outputting the same results as the server 105. In particular, in the case of multiple terminal devices with varying computing power, if the cloud gaming application determines that the terminal device it is in possession of possesses significant computing power and resources, it can allow the terminal device to perform the aforementioned operations, thereby appropriately alleviating the computing pressure on the server 105. Accordingly, the apparatus for delivering the video stream can also be located within the terminal devices 101, 102, and 103.

[0029] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0030] Please refer to Figure 2 , Figure 2 This is a flow chart of a method for delivering a video stream provided by an embodiment of the present disclosure, wherein process 200 includes the following steps:

[0031] Step 201: Determine the target video stream to be sent.

[0032] In this embodiment, the execution subject of the method for sending the video stream (for example Figure 1 The server 105 shown) determines the target video stream to be sent to the target terminal device, wherein the target video stream can be determined according to the acquisition request of the target terminal device, or according to the sending configuration stored in the execution entity and the target video stream determined according to the instructions of the execution entity provider.

[0033] In practice, the target video stream can be the video stream of the video requested by the user, or it can be a video stream composed of game screen content calculated based on the cloud gaming application and sent to the terminal device used by the user for presentation.

[0034] In some embodiments, after determining the target video stream to be sent, the environmental parameters, configuration information, etc. of the target terminal device receiving the target video stream can be optimized according to the obtained environmental parameters and configuration information to obtain an optimized target video stream that adapts to the environmental parameters and configuration information, so as to improve the presentation effect of the target video stream.

[0035] It should be noted that the target video stream can be obtained by the above execution subject directly from a local storage device, or from a non-local storage device (such as Figure 1 The local storage device can be a data storage module provided in the execution subject, such as a server hard disk. In this case, the target video stream can be quickly read locally. The non-local storage device can also be any other electronic device configured to store data, such as some user terminals. In this case, the execution subject can obtain the required target video stream by sending a obtain command to the electronic device.

[0036] Step 202: extract image frames from the target video stream one by one according to a preset interval period to generate a first corrected video stream.

[0037] In this embodiment, after the target video stream is determined in the above step 201, image frames are continuously extracted from the target video stream according to a preset interval period, that is, after the initial image frame in the target video stream is extracted, the next-order image frame is extracted from the target video stream after each interval of the interval period, and then the image frames are arranged into a new video stream, i.e., the first corrected video stream, according to the extraction order of the extracted image frames.

[0038] Among them, the interval period can be determined based on the expected frame interval in the first corrected video, interactively from the side of the target video stream to be sent, or preset according to instructions. The interval period can also be determined accordingly based on the display frame rate of the target terminal device, so that the first corrected video stream obtained according to the interval period is adapted to the frame rate of the target terminal device, thereby improving the sending effect of the target video stream.

[0039] In practice, during the process of extracting image frames, the extracted image frames can be stored in the same video stream sequence in sequence according to the extraction order until the extraction of the last image frame in the target video stream is completed, and the video stream sequence is determined as the first corrected video stream.

[0040] Step 203: Encode the first modified video stream to generate an encoded data stream.

[0041] In this embodiment, after the first modified video stream is generated in step 202, the first modified video stream is encoded using a coding method such as the Moving Picture Expert Group (MPEG) series or the H.26X series to generate a corresponding coded data stream.

[0042] Among them, in some special application scenarios with high requirements for transmission security, encryption can also be performed during the encoding process of the first corrected video stream to obtain an encrypted encoded data stream, so as to improve the security of the video stream being sent down.

[0043] Step 204: Send the encoded data stream to the target terminal device.

[0044] In this embodiment, the coded data stream obtained in 203 is sent to the target terminal device, so that the target terminal device can subsequently decode the coded data stream to obtain the target video stream, thereby completing the video stream distribution work.

[0045] The method for sending a video stream provided by the embodiment of the present disclosure can correct the video stream to be sent by extracting image frames from the target video stream to be sent according to a preset interval period and constructing a corrected video stream during the video stream sending process, so as to avoid frame skipping in the target video stream due to problems such as encoding and transmission delay, and improve the quality of the sent target video stream.

[0046] In some optional implementations of this embodiment, image frames in the target video stream are extracted one by one according to a preset interval period to generate a first corrected video stream, including: extracting image frames in the target video stream one by one according to the interval period; in response to the current extraction operation failing to extract the image frame from the target video stream, using the image frame extracted last time as the image frame extracted this time; in response to each image frame in the target video being extracted at least once, generating a first corrected video stream based on each extracted image frame.

[0047] Specifically, in order to further avoid the lack of image frames or frame skipping in the first corrected video stream due to the missing image frames caused by encoding errors and network transmission errors in the target video stream sent down, in the process of extracting image frames from the target video stream and generating the first video stream, the image frame is extracted this time when the time since the last image frame extraction meets the interval period. If the image frame cannot be extracted from the target video stream this time, the image frame extracted last time is used as the image frame extracted this time, and the extraction time (interval period) is recalculated. Finally, when each image frame in the target video is extracted at least once, a response is performed, and the first corrected video stream is generated based on the extracted image frames. In the case of missing image frames or frame skipping in the target video, the image content can still be displayed normally in the first corrected video stream, thereby improving the stability of the displayed content in the first corrected video stream.

[0048] In some optional implementations of this embodiment, the method for sending the video stream also includes: obtaining the image timestamp information included in the target video stream; determining the time interval between each image frame based on the timestamp information in different image frames; and presetting the time interval with the highest frequency of occurrence as the interval period.

[0049] Specifically, when there is graphic timestamp information corresponding to the image frames in the target video stream, the time interval between each image frame in the target video stream is determined based on the image timestamp information, and the time interval with the highest frequency of occurrence is determined as the reasonable time interval between each image frame in the target video stream, and the time interval is correspondingly preset as the interval period, so that the first corrected video stream generated subsequently can still play the video content according to the time interval of each image frame in the original target video stream, thereby improving the video fidelity during the video stream distribution process.

[0050] In some optional implementations of this embodiment, the method for sending the video stream also includes: obtaining audio timestamp information included in the target video stream; determining the maximum retention time of the image frame based on the audio frame timestamp information; and presetting the interval period according to the maximum retention time of the image frame.

[0051] Specifically, when there is audio timestamp information corresponding to the image frame in the target video stream, the maximum retention time of the image frame can be determined based on the audio frame timestamp information, that is, the time required to complete the playback of the audio information belonging to the image frame, and the interval period can be preset according to the maximum retention time of the image frame to ensure the synchronization of the image and audio in the first corrected video stream.

[0052] Please refer to Figure 3 , Figure 3 This is a flowchart of another method for delivering a video stream provided by an embodiment of the present disclosure, wherein process 300 includes the following steps:

[0053] Step 301: Determine the target video stream to be sent.

[0054] Step 302: extract image frames from the target video stream one by one according to a preset interval period to generate a first corrected video stream.

[0055] Step 303: Encode the first modified video stream to generate an encoded data stream.

[0056] Step 304: Send the encoded data stream to the target terminal device.

[0057] Step 305: Control the target terminal device to decode the encoded data stream and obtain a first corrected video stream.

[0058] In this embodiment, after the coded data stream is sent to the target terminal device based on the above step 304, the target terminal device is controlled to decode the coded data stream to obtain the first modified video stream.

[0059] The execution subject may control and / or instruct the target terminal device to decode the encoded data stream and obtain the corresponding first corrected video stream by directly synchronizing the operation terminal of the target terminal device or sending the encoding rules to the target terminal device.

[0060] Step 306: Control the target terminal device to extract image frames from the first corrected video stream one by one according to the interval period to obtain a second corrected video stream.

[0061] In this embodiment, the above-mentioned execution subject controls the target terminal device to extract image frames from the first corrected video stream one by one according to the interval period to obtain the second corrected video stream. This process is similar to the process of the above-mentioned execution subject extracting image frames from the target video stream locally to obtain the first corrected video stream. By controlling the target terminal device to extract image frames from the first corrected video stream one by one according to the interval period to obtain the second corrected video stream, a secondary correction can be achieved before the target terminal receives the target video stream and plays the target video stream, so as to avoid abnormal image frame intervals during the decoding process of the target terminal device, which affects the playback of the target video stream.

[0062] Among them, the interval period used when generating the second corrected video stream is the same as that when generating the first corrected video stream, but in optional implementation methods in some scenarios, for example, in the scenario where a terminal device other than the cloud server and the target terminal device is used to forward the target video stream from the cloud server to the target terminal device, the interval period used for generating the first corrected video stream and the second corrected video stream can be adjusted according to the data transmission capabilities between the terminal device used for forwarding the target video stream and the cloud server and the target terminal device respectively, so as to achieve the purpose of compressing the first video stream and improving the sending capability of the first video stream by shortening the interval period used for generating the first video stream (that is, when the data transmission capability between the terminal device used for forwarding the target video stream and the cloud server meets the requirements, the first corrected video stream is quickly sent down), so as to ensure that the target terminal device can still play the target video stream at the expected bit rate of the target terminal device by controlling the interval period of generating the second corrected video stream, and quickly complete the sending of the first corrected video stream, thereby reducing frame skipping and playback freeze problems caused by untimely sending of the first corrected video stream.

[0063] The above steps 301-304 are similar to the following Figure 2 The steps 201-204 shown in FIG. 2 are the same as those in FIG. 2 , and the same contents can be found in the corresponding parts of the previous embodiment, which will not be described here. Figure 2 On the basis of the embodiment, this embodiment can further achieve a secondary correction before the target terminal receives the target video stream and plays the target video stream by controlling the target terminal device to extract the image frames in the first corrected video stream one by one according to the interval period to obtain the second corrected video stream, so as to avoid the abnormal interval of the image frame during the decoding process of the target terminal device, which affects the playback of the target video stream.

[0064] To deepen understanding, the present disclosure also provides a specific implementation scheme in combination with a specific application scenario. For easy understanding, the application scenario includes a cloud server A that sends the target video stream, a terminal device B for forwarding the target video stream, and a terminal device C that receives the video stream. Figure 4 Flow 400 is shown.

[0065] Step 401: Determine the target video stream to be delivered.

[0066] Specifically, the terminal device B determines the target video stream in the cloud server A to be sent to the terminal device C, and receives the target video stream sent by the cloud server A.

[0067] Step 402: Generate a first corrected video stream.

[0068] Specifically, after the terminal device B obtains the target video stream, the terminal device B locally extracts image frames in the target video stream one by one according to a preset interval period to generate a first corrected video stream.

[0069] Step 403: Encode the first modified video stream to generate an encoded data stream.

[0070] Specifically, after completing the generation of the first modified video stream locally, the terminal device B encodes the first modified video stream to obtain a corresponding encoded data stream.

[0071] Step 404: Send the encoded data stream to the target terminal device.

[0072] Specifically, terminal device B sends the locally obtained encoded data stream to the target terminal device C, and at the same time sends control information to the terminal device C, so as to instruct the terminal device C through the control information to decode the encoded data stream when it is received, obtain the first corrected data stream corresponding to the encoded data stream, and then extract the image frames in the first corrected video stream one by one according to the preset interval period to obtain the second corrected video stream.

[0073] Step 405: Obtain a second corrected video stream.

[0074] Specifically, after the terminal device C receives the encoded data stream and control information, it decodes the encoded data stream according to the instructions of the control information, obtains the first corrected data stream corresponding to the encoded data stream, and then extracts the image frames in the first corrected video stream one by one according to the preset interval period to obtain the second corrected video stream for playback, completing the video stream sending process from cloud server A to terminal device C.

[0075] Further references Figure 5 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a device for sending a video stream. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0076] like Figure 5As shown, the apparatus 500 for sending a video stream in this embodiment may include: a video stream acquisition unit 501, a first video stream correction unit 502, a coded data stream generation unit 503, and a coded data stream sending unit 504. The video stream acquisition unit 501 is configured to determine a target video stream to be sent; the first video stream correction unit 502 is configured to extract image frames from the target video stream one by one at a preset interval to generate a first corrected video stream; the coded data stream generation unit 503 is configured to encode the first corrected video stream to generate a coded data stream; and the coded data stream sending unit 504 is configured to send the coded data stream to a target terminal device.

[0077] In this embodiment, the specific processing of the video stream obtaining unit 501, the first video stream correction unit 502, the coded data stream generating unit 503 and the coded data stream sending unit 504 and the technical effects thereof can be referred to in the respective Figure 2 The relevant descriptions of steps 201-204 in the corresponding embodiment are not repeated here.

[0078] In some optional implementations of this embodiment, the device 500 for sending the video stream further includes: a decoding control unit, configured to control the target terminal device to decode the encoded data stream and obtain the first corrected video stream; a second video stream correction control unit, configured to control the target terminal device to extract the image frames in the first corrected video stream one by one according to the interval period to obtain the second corrected video stream.

[0079] In some optional implementations of this embodiment, the first video stream correction unit 502 includes: an image frame extraction subunit, configured to extract the image frames in the target video stream one by one according to the interval period; an image frame supplement subunit, configured to use the image frame extracted last time as the image frame extracted this time in response to the current extraction operation failing to extract the image frame from the target video stream; and a correction video stream generation subunit, configured to generate a first correction video stream based on each extracted image frame in response to each image frame in the target video being extracted at least once.

[0080] In some optional implementations of this embodiment, the device 500 for sending the video stream further includes: an image timestamp acquisition unit, configured to obtain the image timestamp information included in the target video stream; an image frame interval determination unit, configured to determine the time interval between each image frame based on the timestamp information in different image frames; and a first interval period preset unit, configured to preset the time interval with the highest frequency of occurrence as the interval period.

[0081] In some optional implementations of this embodiment, the device 500 for sending the video stream further includes: an audio frame interval determination unit, configured to obtain the audio timestamp information included in the target video stream; a maximum holding time determination unit, configured to determine the maximum holding time of the image frame based on the audio frame timestamp information; a second interval period preset unit, configured to preset the interval period according to the maximum holding time of the image frame.

[0082] This embodiment exists as an apparatus embodiment corresponding to the above-mentioned method embodiment. The apparatus for sending a video stream provided by this embodiment can correct the video stream to be sent by extracting image frames from the target video stream to be sent according to a preset interval period and constructing a corrected video stream during the process of sending the video stream, so as to avoid image frame skipping in the target video stream due to problems such as encoding and transmission delay, thereby improving the quality of the sent target video stream.

[0083] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0084] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0085] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the device 600 can also be stored in the RAM 603. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0086] Various components in device 600 are connected to I / O interface 605, including an input unit 606, such as a keyboard, mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a magnetic disk, optical disk, etc.; and a communication unit 609, such as a network card, modem, wireless communication transceiver, etc. The communication unit 609 allows device 600 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0087] The computing unit 601 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as the method for sending a video stream. For example, in some embodiments, the method for sending a video stream can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the method for sending a video stream described above can be performed. Alternatively, in other embodiments, the computing unit 601 can be configured to perform the method for sending a video stream by any other appropriate means (e.g., by means of firmware).

[0088] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0089] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0090] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0091] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0092] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0093] A computer system may include a client and a server. The client and server are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within a cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and virtual private server (VPS) services. Servers may also be classified as distributed system servers or servers integrated with blockchain.

[0094] According to the technical solution of the embodiment of the present disclosure, during the process of sending down the video stream, the video stream to be sent down can be corrected by extracting image frames from the target video stream to be sent down according to a preset interval period and constructing a corrected video stream, so as to avoid frame skipping in the target video stream due to problems such as encoding and transmission delay, thereby improving the quality of the sent target video stream.

[0095] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions provided by this disclosure can be achieved. This is not limited herein.

[0096] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for delivering a video stream, comprising: Determine an initial video stream calculated by the cloud gaming application to be delivered, and optimize the initial target video stream according to environmental parameters and configuration information of a target terminal device that initiates a cloud gaming request to the cloud gaming application to obtain a target video stream; Extracting image frames from the target video stream one by one according to a preset interval period; In response to the current extraction operation failing to extract the image frame from the target video stream, using the image frame extracted last time as the image frame extracted this time; In response to each image frame in the target video being extracted at least once, generating a first modified video stream based on each extracted image frame; wherein the first modified video stream obtained according to the interval period is adapted to a frame rate of the target terminal device; Encoding the first modified video stream to generate a coded data stream; and sending the coded data stream to a target terminal device; The method further includes: acquiring audio timestamp information included in the target video stream; determining a maximum image frame retention time based on the audio timestamp information; and presetting the interval period according to the maximum image frame retention time.

2. The method according to claim 1, further comprising: Controlling the target terminal device to decode the encoded data stream and obtain the first modified video stream; The target terminal device is controlled to extract image frames from the first corrected video stream one by one according to the interval period to obtain a second corrected video stream.

3. The method according to claim 1, further comprising: Obtaining image timestamp information included in the target video stream; determining the time interval between image frames according to the timestamp information in different image frames; The time interval with the highest occurrence frequency is preset as the interval period.

4. A video stream sending device, comprising: a video stream acquisition unit configured to determine an initial video stream to be delivered, calculated by the cloud gaming application, and optimize the initial target video stream according to environmental parameters and configuration information of a target terminal device that initiates a cloud gaming request to the cloud gaming application, so as to obtain a target video stream; An image frame extraction subunit is configured to extract image frames from the target video stream one by one according to a preset interval period; an image frame supplementing subunit, configured to, in response to a failure in extracting the image frame from the target video stream during the current extraction operation, use the image frame extracted last time as the image frame extracted this time; a modified video stream generating subunit, configured to generate a first modified video stream based on each extracted image frame in response to each image frame in the target video being extracted at least once; a coded data stream generating unit, configured to encode the first modified video stream to generate a coded data stream; a coded data stream sending unit, configured to send the coded data stream to a target terminal device; an audio frame interval determining unit, configured to obtain audio timestamp information included in the target video stream; a maximum holding time determining unit, configured to determine a maximum holding time of an image frame based on the audio frame timestamp information; The second interval period preset unit is configured to preset the interval period according to the maximum holding time of the image frame.

5. The apparatus according to claim 4, further comprising: a decoding control unit, configured to control the target terminal device to decode the encoded data stream and obtain the first modified video stream; The second video stream correction control unit is configured to control the target terminal device to extract image frames from the first corrected video stream one by one according to the interval period to obtain a second corrected video stream.

6. The apparatus according to claim 4, further comprising: An image timestamp acquiring unit, configured to acquire image timestamp information included in the target video stream; an image frame interval determining unit, configured to determine the time interval between image frames according to time stamp information in different image frames; The first interval period preset unit is configured to preset the time interval with the highest occurrence frequency as the interval period.

7. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for sending a video stream according to any one of claims 1 to 3.

8. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the method for sending a video stream according to any one of claims 1 to 3.

9. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method for delivering a video stream according to any one of claims 1 to 3 is implemented.

Images