Video stream transmission method, device, equipment and storage medium

Abstract

The present invention discloses a video stream transmission method, device, equipment and storage medium, which are applied to a USB system, and include: a transmitting end and a receiving end. The transmitting end and the receiving end negotiate the video class to determine the default transmission parameters; the transmitting end transmits the video stream to the receiving end based on the default transmission parameters; the receiving end writes the received video stream into a circular buffer area, plays the video stream according to the data volume of the circular buffer area, and determines the video stream playback situation; the transmitting end and the receiving end adjust the default transmission parameters according to the video stream playback situation. By setting the circular buffer area, the receiving end data can be guaranteed to be carried out in an orderly manner, avoiding the memory waste and time-consuming problems caused by data stream duplication. By adjusting the default transmission parameters according to the video stream playback situation, it is possible to dynamically adapt to changes in the network and equipment, optimize the transmission effect, and avoid problems such as freezes and dropped frames. The flexibility of transmission and the quality of video transmission are improved.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a video stream transmission method, device, equipment and storage medium. Background Art

[0002] As a new serial communication technology, USB has many advantages, with constantly updated protocol versions and improved transmission speeds. Different USB device classes need to be enumerated and handshaked before data transmission, and the device type and transmission endpoint are determined through descriptor interaction. Among them, the video class UVC is special, involving synchronous video frame transmission. A large amount of video data stream needs to be transmitted at each time interval without ack confirmation, and high real-time requirements are required.

[0003] In the prior art, the receiving end needs to receive data at a fixed rate within each sof frame time, and after receiving the completed data, the video stream needs to be played at the corresponding rate. When multiple peripherals are accessed or interrupt processing is suddenly time-consuming, the receiving end will cause inconsistent rates during video frame reception and playback, thereby affecting the viewing effect of the video stream. In addition, even if the number of peripherals is small, the receiving end needs to receive the video stream on the one hand and play the video stream on the other hand. There will always be a certain error between the receiving rate and the playback rate. When the error accumulates to a certain extent, there will also be frame drops, freezes, and other situations that affect video viewing. Summary of the invention

[0004] The present invention provides a video stream transmission method, device, equipment and storage medium to eliminate the problems of inconsistent receiving rate and playback rate of the receiver caused by short-term bursts of peripherals in the USB system, avoid data frame loss, freeze and the like, and achieve the purpose of improving the video viewing effect.

[0005] According to one aspect of the present invention, a video stream transmission method is provided, the method comprising:

[0006] The sender and receiver negotiate the video class to determine the default transmission parameters;

[0007] Transmitting a video stream to a receiving end through a sending end based on default transmission parameters;

[0008] The receiving end writes the received video stream into the circular buffer area, plays the video stream according to the data amount of the circular buffer area, and determines the video stream playing status;

[0009] The default transmission parameters are adjusted by the sending end and the receiving end according to the video stream playback situation.

[0010] Optionally, a video class negotiation is performed through the transmitting end and the receiving end to determine the default transmission parameters, including: sending a descriptor to the device end through the host end in the transmitting end and the receiving end; determining the device type corresponding to the received descriptor through the device end, and when the device type is a video class, sending various transmission parameters supported by itself to the host end, wherein the transmission parameters include frame resolution and frame rate; and determining the default transmission parameters from various transmission parameters through the host end.

[0011] Optionally, the video stream is played according to the data volume of the circular buffer, and the playback status of the video stream is determined, including: when the data volume of the circular buffer reaches a preset data volume, reading and playing the video stream in the circular buffer; calculating the remaining space of the circular buffer in real time; judging whether the remaining space meets the preset range, and if so, determining that the playback status of the video stream is normal; otherwise, determining that the playback status of the video stream is abnormal.

[0012] Optionally, the remaining space of the circular buffer is calculated in real time, including: determining the read pointer position and the write pointer position of the circular buffer, and determining the pointer difference between the read pointer position and the write pointer position, and using the pointer difference as the cached data amount; obtaining the total space data amount of the circular buffer, and calculating the difference between the total space data amount and the cached data amount to determine the remaining space.

[0013] Optionally, the default transmission parameters are adjusted by the transmitter and the receiver according to the video stream playback situation, including: when the video stream playback situation is abnormal, the transmitter and the receiver renegotiate the default transmission parameters according to the remaining space to determine the updated transmission parameters.

[0014] Optionally, after the default transmission parameters are adjusted according to the video stream playback situation by the sending end and the receiving end, the method also includes: transmitting the video stream to the receiving end based on the updated transmission parameters by the sending end, and monitoring the video stream playback situation; when the video stream playback situation returns to normal playback, renegotiating through the sending end and the receiving end to restore the updated transmission parameters to the default transmission parameters.

[0015] Optionally, the method also includes: when the receiving end is a host end, sending a specified command to the sending end to negotiate the transmission parameters; when the receiving end is a device end, sending parameter update information to the sending end through an interrupt endpoint to negotiate the transmission parameters.

[0016] According to another aspect of the present invention, a video stream transmission device is provided, the device comprising:

[0017] A default transmission parameter determination module, used for performing video class negotiation through a transmitting end and a receiving end to determine a default transmission parameter;

[0018] A video stream transmission module, used for transmitting a video stream from a sending end to a receiving end based on default transmission parameters;

[0019] The video stream playback status determination module is used to write the received video stream into the circular buffer area through the receiving end, play the video stream according to the data volume of the circular buffer area, and determine the video stream playback status;

[0020] The transmission parameter adjustment module is used to adjust the default transmission parameters according to the video stream playback situation through the sending end and the receiving end.

[0021] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0022] at least one processor;

[0023] and a memory communicatively coupled to the at least one processor;

[0024] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a video stream transmission method described in any embodiment of the present invention.

[0025] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a video stream transmission method according to any embodiment of the present invention when executed.

[0026] The technical solution of the embodiment of the present invention can ensure that the data at the receiving end is carried out in an orderly manner by setting a circular buffer area, avoiding the memory waste and time-consuming problems caused by data stream duplication. By adjusting the default transmission parameters according to the video stream playback situation, it can dynamically adapt to changes in the network and equipment, optimize the transmission effect, and avoid problems such as freezes and dropped frames. The flexibility and adaptability of transmission are improved, ensuring that better video transmission quality can be provided in different environments.

[0027] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 is a flowchart of a video stream transmission method provided according to Embodiment 1 of the present invention;

[0030] Figure 2 is a flowchart of another video stream transmission method provided according to Embodiment 1 of the present invention;

[0031] Figure 3 is a schematic diagram of the structure of a ring buffer area provided according to the first embodiment of the present invention;

[0032] Figure 4 is a flowchart of another video stream transmission method provided according to Embodiment 2 of the present invention;

[0033] Figure 5 is a structural schematic diagram of a video stream transmission device provided according to Embodiment 3 of the present invention;

[0034] Figure 6 It is a structural schematic diagram of an electronic device for implementing a video stream transmission method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0036] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0037] Embodiment 1

[0038] Figure 1A flowchart of a video stream transmission method is provided for the first embodiment of the present invention. This embodiment is applicable to the scenario of transmitting video streams in a USB system. The method can be executed by a video stream transmission device. The video stream transmission device can be implemented in the form of hardware and / or software. The video stream transmission device can be configured in a computer controller. Figure 1 As shown, the method includes:

[0039] S110: The sending end and the receiving end perform video-related negotiation to determine default transmission parameters.

[0040] In the USB system, the sender is the device responsible for encoding, processing, and sending video data. The receiver is the device that corresponds to the sender and is used to receive video data from the sender and perform operations such as decoding and playing it. The default transmission parameters refer to the basic transmission settings negotiated and determined by the sender and the receiver before starting to transmit the video, including video resolution and frame rate, that is, the number of frames displayed per second.

[0041] Specifically, at the initial stage of video transmission, the sender and the receiver need to reach an initial agreement on transmission, that is, to determine the default transmission parameters. Different videos have different characteristics. The sender needs to know the video format, resolution range, and frame rate supported by the receiver, and the receiver must also know the general situation of the video to be sent by the sender. After negotiation, the two parties find a default parameter combination that is compatible with each other and suitable for this video transmission, such as starting the transmission with 1080P resolution and 30 frames per second. It is like two people agreeing on a common "language" and "rules" to communicate video data.

[0042] Optionally, a video class negotiation is performed through the transmitting end and the receiving end to determine the default transmission parameters, including: sending a descriptor to the device end through the host end in the transmitting end and the receiving end; determining the device type corresponding to the received descriptor through the device end, and when the device type is a video class, sending various transmission parameters supported by itself to the host end, wherein the transmission parameters include frame resolution and frame rate; and determining the default transmission parameters from various transmission parameters through the host end.

[0043] It should be noted that in the preparation stage of video transmission, the host side in the sending and receiving ends plays the role of initiating communication. The host side sends a descriptor to the device side in order to initiate a request for information to the device side. The descriptor may contain basic attributes such as the general application scenario and source of this video transmission, so that the device side can accurately know what kind of video-related transactions to be processed next, and can also make subsequent corresponding response operations based on the descriptor. After receiving the descriptor sent by the host side, the device side will parse and judge it. The device side determines the corresponding device type by analyzing the key feature information carried in the descriptor. Only when it is clear that it is a video-related UVC requirement will it perform subsequent operations on video transmission.

[0044] Specifically, when it is determined to be a video device, the device will send the transmission parameters it supports to the host. Taking common video playback devices as an example, different display screens, chip processing capabilities and other factors determine the frame resolution and frame rate it can support. For example, some devices can support up to 4K, i.e. 3840×2160 resolution, while some can only support up to 1080P, etc. Some can smoothly play videos at 60 frames per second, while some can only play up to 30 frames per second. The transmission parameters reflect the device's capabilities in video processing and display. By sending the transmission parameters it supports to the host, the device is informing the host of "what quality and specifications of video data it can receive", so that the host has a reference for determining subsequent parameters.

[0045] Furthermore, the host side will select the default transmission parameters. The host side will consider many factors. On the one hand, it will consider the original characteristics of the video source itself. For example, if the original video itself is shot at 4K resolution and 60 frames per second, it will give priority to whether the device side can support such high-specification parameters; on the other hand, it will also consider the current network conditions, system resources and other external conditions. If the network bandwidth is limited, even if the device side supports high resolution and high frame rate, forcibly using such parameter transmission may cause frequent freezes or even transmission failures. Therefore, the host side will weigh and select a parameter combination that can be well played and presented on the device side and can adapt to external conditions such as the network as the default transmission parameter to ensure that the entire video transmission and playback process can proceed smoothly.

[0046] S120: Transmitting the video stream to the receiving end through the sending end based on the default transmission parameters.

[0047] The video stream refers to a continuous data sequence formed by segmenting and packaging the video data in a time sequence and a specific format. The sender will process and send the original video data according to the default transmission parameters.

[0048] S130, writing the received video stream into the circular buffer area through the receiving end, playing the video stream according to the data amount of the circular buffer area, and determining the video stream playing status.

[0049] Among them, the circular buffer is a data storage area with a structure similar to a ring. When the data fills the entire area, the new data will overwrite the earliest stored data. It is used at the receiving end to balance the instability of network transmission and the continuity of data reading at the playback end.

[0050] Specifically, after receiving the video stream data, the receiving end will first temporarily store the video stream in the circular buffer. The data in the circular buffer will be continuously stored and retrieved for recycling. By setting up the circular buffer, you can balance the instability of network transmission and the continuity requirements of the player to read data. For example, network transmission may experience short-term freezes or data bursts. If there is no buffer, the playback will be frequently interrupted or jammed. The receiving end will play the video stream according to the amount of data accumulated in the buffer. When the data volume reaches a certain level, it is enough to ensure that the playback will not be interrupted for a short period of time, and then it will start playing. At the same time, it will continuously monitor whether there are bursts and error accumulation to determine the video stream playback status.

[0051] Figure 2 A flowchart of a video stream transmission method is provided for the first embodiment of the present invention, wherein step S130 mainly includes the following steps S131 to S136:

[0052] S131. The receiving end writes the received video stream into the ring buffer.

[0053] S132: When the data volume of the circular buffer reaches a preset data volume, the video stream of the circular buffer is read and played.

[0054] It is known that video playback requires sufficient continuous data to maintain normal display of the picture and synchronous output of audio, etc. The preset data volume can be set to 50% of the circular buffer space. Only when the preset data volume is accumulated, the receiving end will have enough "materials" to decode, render the picture, and play the sound in sequence, so as to avoid interruption and jamming due to insufficient data at the beginning of playback. When the accumulated data volume of the circular buffer reaches the preset data volume, the playback program will read the video stream data from the circular buffer in sequence for playback.

[0055] In a specific embodiment, Figure 3 A schematic diagram of the structure of a circular buffer is provided for the first embodiment of the present invention, wherein Bank_0 to Bank_N-1 are storage blocks in the buffer. In the circular buffer, data is stored in the storage blocks in sequence. Figure 3The start and end positions of the buffer area are indicated in the figure. The start address points to Bank_0, and the end address points to Bank_N-1. And after reaching the end address, the next address will loop back to Bank_0 to form a ring structure. For example, when the first data enters, it will be stored in Bank_0, the second data will be stored in Bank_1, and so on, until it is stored in Bank_N-1. When Bank_N-1 is reached, the next data will return to Bank_0 for storage. p_read is the pointer for reading / playing the data stream. When reading data from the buffer area, the data position is located through the p_read pointer. After each data is read, the p_read pointer automatically points to the next position (+1 operation). p_write is the pointer for writing data streams. When writing data to the buffer area, the data write position is located through the p_write pointer. After each data is written, the p_write pointer automatically points to the next position (+1 operation). In specific applications, after the receiving end configures the USB controller receiving parameters, the hardware will write the corresponding data stream to the circular array p_write (the write address is software configurable), and automatically update the p_write position and automatically add 1 after writing the data. If p_write is the last bank in the buffer, it will jump to the beginning of the buffer, forming a loop; when the video stream in the circular buffer exceeds 50% of the buffer capacity, the playback device starts reading the video stream data from p_read (starting position is 0) and starts playing the video. After the playback is completed, the p_read position will also increase by 1. Through this circular buffer, data can be read and written effectively without additional data relocation, reducing processing time.

[0056] S133, calculating the remaining space of the circular buffer in real time.

[0057] During video playback, it is necessary to keep track of the remaining space in the circular buffer in real time. On the one hand, the video stream is continuously stored in the circular buffer, and at the same time, the player is continuously reading data from the buffer for playback. The above two processes are dynamically parallel. Through a specific algorithm, the system can count at any time how much space is left in the current buffer to continue to store the newly incoming video stream data.

[0058] Optionally, the remaining space of the circular buffer is calculated in real time, including: determining the read pointer position and the write pointer position of the circular buffer, and determining the pointer difference between the read pointer position and the write pointer position, and using the pointer difference as the cached data amount; obtaining the total space data amount of the circular buffer, and calculating the difference between the total space data amount and the cached data amount to determine the remaining space.

[0059] The ring buffer is logically presented as a ring structure, in which data is stored and read out in a certain order. In order to effectively manage and track the flow of data, this implementation introduces two key elements, the read pointer and the write pointer, and specifies the following parameters: The size of a bank in the ring buffer: buffer bank ; The entire ring buffer space: buffer total =buffer bank *N; remaining space in the ring buffer: buffer remainer =buffer bank *N-|p_write–p_read|; where buffer total Indicates the size of the entire circular buffer, buffer remainer The remaining writable free space in the entire circular buffer area (or the data has been played and can be overwritten with new data).

[0060] Specifically, the read pointer is used to indicate the location where data is currently being read from the circular buffer, while the write pointer marks the location where new video stream data is being written to the circular buffer. Due to the circular structure, the two pointers will move continuously and cyclically, and the difference between them can be calculated to know how much data has been stored in the current buffer. Then, the remaining space can be determined by calculating the difference between the total space data volume and the cache data volume.

[0061] S134, determine whether the remaining space meets the preset range, if so, execute S135, otherwise, execute S136.

[0062] S135: Determine whether the video stream playback is normal.

[0063] It can be known that a suitable remaining space range means that the storage and retrieval of data are in a good balance. Therefore, the preset range of the remaining space in the circular buffer is a key indicator to ensure smooth video playback. If the remaining space is within the preset range, for example, the preset range is that the remaining space accounts for 25%-75% of the total space, it means that the current video stream storage speed and the reading speed of the playback end are properly matched, and there will be no backlog of data causing the buffer to be filled up quickly, nor will there be too little data in the buffer due to too fast reading, resulting in playback interruption. Therefore, the video stream playback is judged to be normal playback at this time.

[0064] S136: Determine that the video stream playback status is abnormal.

[0065] Specifically, when the remaining space exceeds the preset range, for example, the remaining space is less than 25%, it means that the playback end reads data too fast, and it is likely that there will be a problem of playback freeze or even interruption due to a lack of data in the cache. If the remaining space is greater than 75%, it means that the data is stored too slowly or the playback end has a reading failure, and a large amount of data is accumulated in the cache, which also reflects that there is an abnormality in the video playback, so the video stream playback will be judged as abnormal playback.

[0066] S140: The sending end and the receiving end adjust the default transmission parameters according to the video stream playback situation.

[0067] Specifically, the receiving end will feed back the video stream playback status to the sending end. After receiving the feedback, the sending end will analyze the cause. If it is caused by insufficient network bandwidth, the video resolution or frame rate may be reduced to reduce the amount of data transmission, so that the video can be played more smoothly; if the playback is smooth and the receiving end has sufficient processing power and network bandwidth margin, the resolution and frame rate can also be appropriately increased to improve the video viewing quality.

[0068] The technical solution of the embodiment of the present invention can ensure that the data at the receiving end is carried out in an orderly manner by setting a circular buffer area, avoiding the memory waste and time-consuming problems caused by data stream duplication. By adjusting the default transmission parameters according to the video stream playback situation, it can dynamically adapt to changes in the network and equipment, optimize the transmission effect, and avoid problems such as freezes and dropped frames. The flexibility and adaptability of transmission are improved, ensuring that better video transmission quality can be provided in different environments.

[0069] Embodiment 2

[0070] Figure 4 This is a flowchart of a video stream transmission method provided in Embodiment 2 of the present invention. This embodiment adds a specific process of adjusting the default transmission parameters by the transmitter and the receiver according to the video stream playback situation on the basis of the above Embodiment 1. Among them, the specific contents of steps S210-S230 are roughly the same as steps S110-S130 in Embodiment 1, so they will not be repeated in this embodiment. Figure 4 As shown, the method includes:

[0071] S210: The sending end and the receiving end perform video-related negotiation to determine default transmission parameters.

[0072] Optionally, a video class negotiation is performed through the transmitting end and the receiving end to determine the default transmission parameters, including: sending a descriptor to the device end through the host end in the transmitting end and the receiving end; determining the device type corresponding to the received descriptor through the device end, and when the device type is a video class, sending various transmission parameters supported by itself to the host end, wherein the transmission parameters include frame resolution and frame rate; and determining the default transmission parameters from various transmission parameters through the host end.

[0073] S220: Transmitting the video stream to the receiving end through the sending end based on the default transmission parameters.

[0074] S230, writing the received video stream into the circular buffer area through the receiving end, playing the video stream according to the data amount of the circular buffer area, and determining the video stream playing status.

[0075] Optionally, the video stream is played according to the data volume of the circular buffer, and the playback status of the video stream is determined, including: when the data volume of the circular buffer reaches a preset data volume, reading and playing the video stream in the circular buffer; calculating the remaining space of the circular buffer in real time; judging whether the remaining space meets the preset range, and if so, determining that the playback status of the video stream is normal; otherwise, determining that the playback status of the video stream is abnormal.

[0076] Optionally, the remaining space of the circular buffer is calculated in real time, including: determining the read pointer position and the write pointer position of the circular buffer, and determining the pointer difference between the read pointer position and the write pointer position, and using the pointer difference as the cached data amount; obtaining the total space data amount of the circular buffer, and calculating the difference between the total space data amount and the cached data amount to determine the remaining space.

[0077] S240: When the video stream playback situation is abnormal, the sending end and the receiving end renegotiate the default transmission parameters according to the remaining space to determine the updated transmission parameters.

[0078] Among them, when the video stream playback is determined to be abnormal, it indicates that there is a problem with the current video transmission and playback, such as screen freeze, frame loss or playback interruption. The reason why the default transmission parameters are renegotiated based on the remaining space in the circular buffer is that the state of the remaining space in the circular buffer can reflect the balance between the storage and reading of video stream data, and is further related to whether the transmission link is adapted. By re-exchanging information between the sender and the receiver and weighing various factors, the default transmission parameters such as frame rate, resolution, bit rate and other key elements are adjusted to determine the updated transmission parameters, so that the video stream can be subsequently transmitted and played in a more appropriate state, improving the abnormal playback situation.

[0079] Specifically, if the remaining space in the circular buffer is too small, for example, the proportion of the remaining space to the total space is far below the reasonable range, it is likely that the speed at which the receiving end reads the video stream data is too fast, exceeding the speed at which new data is stored. At this time, it is easy to appear that there is no data available at the end of the playback and the phenomenon of freezing. This situation may be because the default transmission parameters initially set have too high a frame rate and too large a resolution, resulting in too much data transmitted per unit time, and too high requirements for resources such as network bandwidth, so that new data cannot be added to the buffer in time. At this time, the sender and the receiver need to renegotiate the transmission parameters. The sender should consider reducing the frame rate of the video, such as from the original 60 frames / second to 30 frames / second, or reducing the resolution, such as from 4K to 1080P, etc., to reduce the amount of data per frame and the total amount of data transmitted per unit time, so that new data can be stored in the circular buffer more smoothly and keep up with the reading speed of the receiver.

[0080] On the contrary, if the remaining space in the circular buffer is too large and exceeds the reasonable range, it means that the receiving end reads data too slowly, which may be because the data transmission volume is too small under the current transmission parameter settings, and it is in a state of waiting for data for a long time. Then when the two parties renegotiate, the sender can appropriately increase the frame rate, resolution and other parameters to increase the data transmission volume, so that the receiving end has enough data for smooth playback, and at the same time, it can make more reasonable use of network resources.

[0081] Optionally, the method also includes: when the receiving end is a host end, sending a specified command to the sending end to negotiate the transmission parameters; when the receiving end is a device end, sending parameter update information to the sending end through an interrupt endpoint to negotiate the transmission parameters.

[0082] In a specific implementation, if the receiving end is a host end, the frame rate and resolution can be adjusted by sending a specified command to the sending end. The specified command can be vs_probe_control, vs_commit_control or suspend command. If the receiving end is a device end, some parameter change information can be sent to the host end through an interrupt endpoint, so that the host end can renegotiate the rate until the remaining space is within a reasonable range, and then restore to the previous default transmission parameters according to user needs.

[0083] S250: The sending end transmits the video stream to the receiving end based on the updated transmission parameters, and monitors the playback status of the video stream.

[0084] S260: When the video stream playback status returns to normal playback, the transmitting end and the receiving end renegotiate to restore the updated transmission parameters to the default transmission parameters.

[0085] Specifically, when the sender and the receiver adjust the default transmission parameters according to the abnormal situation of the previous video stream playback and determine the updated transmission parameters, the sender will re-encode and encapsulate the original video data according to the updated transmission parameters, and continue to monitor the video stream playback. When the video stream playback is monitored to return to normal playback, it indicates that the current video transmission and playback have reached a relatively ideal state, and the video can be presented to the user smoothly.

[0086] It should be noted that updating the transmission parameters is an adjustment made in case of abnormal playback in order to adapt to the poor transmission conditions at the time. It is a temporary compromise solution, and its purpose is to allow the video to play normally first. The default transmission parameters are usually determined during the initial negotiation, taking into account multiple factors such as the quality of the video source itself, the optimal presentation capability of the receiving device, and the general network conditions, and are more in line with the overall requirements. For example, the original video itself is high-quality 4K resolution, 60 frames per second content, and the receiving device is fully capable of playing such high-quality videos smoothly. Previously, the parameters were lowered only because of temporary network fluctuations and other reasons. Now that the playback is normal, the two parties can renegotiate, the sender informs the receiver to prepare for recovery, the receiver confirms that it can adapt, and then gradually adjusts the parameters back to the original default transmission parameters, while ensuring the viewing experience while making full use of resources such as equipment and network.

[0087] The technical solution of the embodiment of the present invention can solve the problems in transmission in a targeted manner by determining and updating the transmission parameters according to the remaining space, improve the accuracy and effectiveness of the adjustment, and can promptly respond to abnormal video stream playback and quickly take measures to adjust the parameters to avoid the problem from continuing to affect the transmission effect. When it is monitored that the video stream playback situation has returned to normal playback, the parameters are adjusted back to the original default transmission parameters, while ensuring the viewing experience, while also making full use of resources such as equipment and network. It can dynamically adapt to changes in the network and equipment, optimize the transmission effect, and avoid problems such as freezes and dropped frames.

[0088] Embodiment 3

[0089] Figure 5 FIG. 1 is a schematic diagram of the structure of a video stream transmission device provided in Embodiment 3 of the present invention. Figure 5 As shown, the device includes: a default transmission parameter determination module 310, which is used to perform video-type negotiation through a transmitting end and a receiving end to determine a default transmission parameter;

[0090] The video stream transmission module 320 is used to transmit the video stream to the receiving end through the sending end based on the default transmission parameters;

[0091] The video stream playing condition determination module 330 is used to write the received video stream into the circular buffer area through the receiving end, play the video stream according to the data amount of the circular buffer area, and determine the video stream playing condition;

[0092] The transmission parameter adjustment module 340 is used to adjust the default transmission parameters according to the video stream playback situation through the sending end and the receiving end.

[0093] Optionally, the default transmission parameter determination module 310 is specifically used to: send a descriptor to the device side through the host side in the sending end and the receiving end; determine the device type corresponding to the received descriptor through the device side, and when the device type is a video class, send each transmission parameter supported by itself to the host side, wherein the transmission parameters include frame resolution and frame rate; determine the default transmission parameter from each transmission parameter through the host side.

[0094] Optionally, the video stream playback status determination module 330 specifically includes: a video stream playback status determination unit, which is used to: when the data volume of the circular buffer area reaches a preset data volume, read and play the video stream in the circular buffer area; calculate the remaining space of the circular buffer area in real time; determine whether the remaining space meets the preset range, and if so, determine that the video stream playback status is normal; otherwise, determine that the video stream playback status is abnormal.

[0095] Optionally, the video stream playback status determination unit specifically includes: a remaining space determination sub-unit, used to: determine the read pointer position and the write pointer position of the circular buffer area, and determine the pointer difference between the read pointer position and the write pointer position, and use the pointer difference as the cached data amount; obtain the total space data amount of the circular buffer area, and calculate the difference between the total space data amount and the cached data amount to determine the remaining space.

[0096] Optionally, the transmission parameter adjustment module 340 is specifically used to: when the video stream playback situation is abnormal, the sending end and the receiving end renegotiate the default transmission parameters according to the remaining space to determine the updated transmission parameters.

[0097] Optionally, the device also includes: a transmission parameter recovery module, which is used to transmit the video stream to the receiving end based on the updated transmission parameters through the sending end, and monitor the playback status of the video stream after the default transmission parameters are adjusted according to the video stream playback status through the sending end and the receiving end; when the video stream playback status returns to normal playback, the sending end and the receiving end renegotiate to restore the updated transmission parameters to the default transmission parameters.

[0098] Optionally, the device also includes: a transmission parameter negotiation module, which is used to negotiate transmission parameters by sending specified commands to the sending end when the receiving end is a host end; when the receiving end is a device end, send parameter update information to the sending end through an interrupt endpoint to negotiate transmission parameters.

[0099] The technical solution of the embodiment of the present invention can ensure that the data at the receiving end is carried out in an orderly manner by setting a circular buffer area, avoiding the memory waste and time-consuming problems caused by data stream duplication. By adjusting the default transmission parameters according to the video stream playback situation, it can dynamically adapt to changes in the network and equipment, optimize the transmission effect, and avoid problems such as freezes and dropped frames. The flexibility and adaptability of transmission are improved, ensuring that better video transmission quality can be provided in different environments.

[0100] A video stream transmission device provided in an embodiment of the present invention can execute a video stream transmission method provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0101] Embodiment 4

[0102] Figure 6 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0103] like Figure 6 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0104] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0105] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a video streaming method.

[0106] In some embodiments, a video streaming method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the video streaming method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform a video streaming method in any other appropriate manner (e.g., by means of firmware).

[0107] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0108] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0109] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0110] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0111] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0112] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0113] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0114] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A video stream transmission method, characterized in that: Applied to a USB system, comprising: a transmitting end and a receiving end, the method comprises: The sender and receiver negotiate the video class to determine the default transmission parameters; Transmitting the video stream to the receiving end based on the default transmission parameters by the sending end; Writing the received video stream into a circular buffer area through a receiving end, playing the video stream according to the data amount of the circular buffer area, and determining the video stream playing status; The default transmission parameters are adjusted by the sending end and the receiving end according to the video stream playing situation.

2. The method according to claim 1, characterized in that The video class negotiation is performed by the transmitting end and the receiving end to determine the default transmission parameters, including: Sending descriptors to the device through the host side in the transmitting side and the receiving side; Determine the device type corresponding to the received descriptor through the device side, and when the device type is a video class, send various transmission parameters supported by the device side to the host side, wherein the transmission parameters include frame resolution and frame rate; The host side determines the default transmission parameters from the transmission parameters.

3. The method according to claim 1, characterized in that Playing the video stream according to the data amount of the annular buffer and determining the video stream playing status includes: When the data volume of the circular buffer reaches a preset data volume, reading and playing the video stream of the circular buffer; Calculating the remaining space of the circular buffer in real time; Determine whether the remaining space meets a preset range, and if so, determine that the video stream playback is normal; Otherwise, it is determined that the video stream playback situation is abnormal playback.

4. The method according to claim 3, characterized in that The real-time calculation of the remaining space of the ring buffer includes: Determine a read pointer position and a write pointer position of the annular buffer area, determine a pointer difference between the read pointer position and the write pointer position, and use the pointer difference as the buffer data amount; The total space data volume of the circular buffer area is obtained, and the difference between the total space data volume and the buffer data volume is calculated to determine the remaining space.

5. The method according to claim 3, characterized in that: The adjusting the default transmission parameters according to the video stream playback situation by the transmitting end and the receiving end includes: When the video stream playback situation is abnormal, the default transmission parameters are renegotiated by the sending end and the receiving end according to the remaining space to determine the updated transmission parameters.

6. The method according to claim 5, characterized in that After the default transmission parameters are adjusted by the transmitting end and the receiving end according to the video stream playing situation, the method further includes: Transmitting the video stream to the receiving end through the sending end based on the updated transmission parameters, and monitoring the playback status of the video stream; When the video stream playback status returns to normal playback, the updated transmission parameters are restored to the default transmission parameters through renegotiation between the sending end and the receiving end.

7. The method according to claim 2, characterized in that The method further comprises: When the receiving end is a host end, a specified command is sent to the sending end to negotiate transmission parameters; When the receiving end is a device end, parameter update information is sent to the sending end through an interrupt endpoint to negotiate transmission parameters.

8. A video stream transmission device, applied to the method according to any one of claims 1 to 7, characterized in that: include: A default transmission parameter determination module, used for performing video class negotiation through a transmitting end and a receiving end to determine a default transmission parameter; A video stream transmission module, used for transmitting the video stream to the receiving end through the sending end based on the default transmission parameters; A video stream playback status determination module, used to write the received video stream into a circular buffer area through a receiving end, play the video stream according to the data volume of the circular buffer area, and determine the video stream playback status; The transmission parameter adjustment module is used to adjust the default transmission parameters according to the video stream playback situation through the sending end and the receiving end.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that: The computer storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method according to any one of claims 1 to 7 when executed.

Images