Video fragment transmission method and device and video fragment receiving method and device

Abstract

The invention discloses a video fragment transmission method and device and a video fragment receiving method and device, and relates to the technical field of communication, and the method comprises the steps: carrying out the fragmentation of a target video, and obtaining a fragment set containing N video fragments, and creating a plurality of data streams based on the QUIC protocol according to the number of the fragments, the buffer limit requirements and the equipment resources, transmitting the fragment set to the server by adopting the plurality of data streams, transmitting video fragments by adopting a time-sharing alternating strategy for each data stream in the plurality of data streams, and transmitting video fragments by adopting a concurrent strategy for the plurality of data streams. According to the invention, the technical problems that a fixed number of data streams are adopted to transmit video fragments in the prior art, network fluctuation cannot be handled, and the limitation is relatively strong are solved.

Description

Technical Field

[0001] The present invention relates to the field of audio and video communication technology or other related fields, and in particular to a method and device for transmitting video slices, and a method and device for receiving video slices. Background Art

[0002] With the popularity of high-definition and 4K videos, the demand for video transmission is growing. The traditional TCP protocol (Transmission Control Protocol) has problems such as high connection establishment delay and low transmission efficiency in video transmission, especially in weak network environments.

[0003] In the related art, the video transmission scheme based on the QUIC protocol mainly focuses on the transmission of video streams, and does not provide sufficient support for uploading video segments. At the same time, the video transmission scheme in the prior art uses multiplexing to transmit audio and video data, but creates a fixed number of data streams (streams) that cannot cope with network fluctuations and does not consider the resource limitations of embedded systems, which has strong limitations.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a method and device for transmitting video fragments, and a method and device for receiving video fragments, so as to at least solve the technical problem in the related art that a fixed number of data streams are used to transmit video fragments, which cannot cope with network fluctuations and has strong limitations.

[0006] According to one aspect of an embodiment of the present invention, a method for transmitting video fragments is provided, which is applied to a video sending end, and the method for transmitting video fragments includes: fragmenting a target video to obtain a fragment set including N video fragments, where N is a positive integer; creating multiple data streams based on the QUIC protocol according to the number of fragments, buffer limit requirements and device resources, wherein the buffer limit requirements include: the maximum transmission fragment allowed to be occupied by the buffer and the number of transmittable fragments, and the device resources include: CPU resources, memory resources and cache resources that can be used by the multiple data streams when transmitting video fragments; using the multiple data streams to transmit the fragment set to a server, wherein each of the multiple data streams adopts a time-sharing alternating strategy to transmit the video fragments, and the multiple data streams adopt a concurrent strategy to transmit the video fragments.

[0007] Optionally, the use of the multiple data streams to transmit the segment set to the server includes: when the data stream recognizes the segment end identifier of the video segment, marking the integrity of the video segment as true and recording the segment length; or, when the data stream is abnormally closed or the response transmission operation times out, marking the integrity of the video segment as no and recording the segment length.

[0008] Optionally, in the process of using the multiple data streams to transmit the fragment set to the server, it also includes: after the data stream receives the fragment upload request from the video sender, it is determined that the fragment sequence number in the fragment upload request already exists and the fragment integrity is yes, and the fragment upload request is rejected; or, after the data stream receives the fragment upload request from the video sender, it is determined that the fragment sequence number in the fragment upload request already exists, but the fragment integrity is no, then the fragment upload request is received, the video fragment in the fragment upload request is extracted, the fragment length of the extracted video fragment is compared with the fragment length of the fragment with the same fragment sequence number received in the historical process, and the video fragment with the largest fragment length is retained.

[0009] Optionally, the video fragment transmission method also includes: monitoring the CPU usage, memory usage, the number of streams of all current data streams, and cache usage of the video sending end; creating a new data stream when the cache usage is lower than the cache usage threshold, the CPU usage is lower than the CPU usage threshold, and the memory usage is lower than the memory usage threshold; stopping the creation of the data stream when the cache usage is greater than or equal to the cache usage threshold, the CPU usage is greater than or equal to the CPU usage threshold, and the memory usage is greater than or equal to the memory usage threshold.

[0010] Optionally, the use of the multiple data streams to transmit the segment set to the server also includes: when a stream end signal is detected, triggering a destruction instruction, wherein the destruction instruction is used to close the data stream and release CPU resources and memory resources; when a transmission error or transmission timeout occurs in the process of using the data stream to transmit the video segment, closing the data stream, recreating a new data stream, and using the new data stream to transmit the video segment.

[0011] Optionally, the method for transmitting video fragments also includes: when it is monitored that the transmission of all the video fragments is completed, sending an upload end request to the server, wherein the server, when receiving the upload end request, merges all the video fragments and saves the target video.

[0012] Optionally, after creating multiple data streams based on the QUIC protocol, it also includes: assigning a stream identifier to each of the data streams, and the stream identifiers corresponding to the various data streams are different.

[0013] According to another aspect of an embodiment of the present invention, a method for receiving video fragments is also provided, which is applied to a server, and the method for receiving video fragments includes: receiving a fragment set transmitted by a video sending end, wherein the fragment set is obtained using any one of the above-mentioned video fragment transmission methods; merging all video fragments in the fragment set according to an integrity priority strategy or a continuity priority strategy, wherein the integrity priority strategy includes: retaining all received video fragments and merging them in ascending order of fragment sequence numbers; the continuity priority strategy includes: dividing all video fragments into fragment groups according to a maximum discontinuity threshold, selecting the fragment group with the largest number, obtaining a fragment sequence number list to be retained based on the fragment sequence number of each video fragment in the selected fragment group, and merging all retained video fragments in ascending order of the sequence numbers in the fragment sequence number list.

[0014] According to another aspect of an embodiment of the present invention, there is further provided a transmission device for video fragments, which is applied to a video sending end, and the transmission device for video fragments includes: a video fragmentation unit, which is used to fragment the target video to obtain a fragment set including N video fragments, where N is a positive integer; a data stream creation unit, which is used to create multiple data streams based on the QUIC protocol according to the number of fragments, buffer limit requirements and device resources, wherein the buffer limit requirements include: the maximum transmission fragment allowed to be occupied by the buffer and the number of transmittable fragments, and the device resources include: CPU resources, memory resources and cache resources that can be used by the multiple data streams when transmitting video fragments; a fragment transmission unit, which is used to use the multiple data streams to transmit the fragment set to the server end, wherein each of the multiple data streams adopts a time-sharing alternating strategy to transmit the video fragments, and the multiple data streams adopt a concurrent strategy to transmit the video fragments.

[0015] Optionally, the fragment transmission unit includes: a first fragment marking module, used to mark the integrity of the video fragment as true and record the fragment length when the data stream recognizes the fragment end mark of the video fragment; or, a second fragment marking module, used to mark the integrity of the video fragment as no and record the fragment length when the data stream is abnormally closed or the response transmission operation times out.

[0016] Optionally, the segment transmission unit also includes: a first determination module, used to determine, after the data stream receives the segment upload request from the video sending end, that the segment sequence number in the segment upload request already exists and the segment integrity is yes, and reject the segment upload request; or, a second determination module, used to determine, after the data stream receives the segment upload request from the video sending end, that the segment sequence number in the segment upload request already exists but the segment integrity is no, then receive the segment upload request, extract the video segment in the segment upload request, compare the segment length of the extracted video segment with the segment length of the same segment sequence number received in the historical process, and retain the video segment with the maximum segment length.

[0017] Optionally, the transmission device of video fragments also includes: a resource monitoring unit, which is used to monitor the CPU usage, memory usage, the number of streams of all current data streams, and cache usage of the video sending end; a new data stream creation unit, which is used to create a new data stream when the cache usage is lower than the cache usage threshold, the CPU usage is lower than the CPU usage threshold, and the memory usage is lower than the memory usage threshold; and a data stream creation stop unit, which is used to stop creating data streams when the cache usage is greater than or equal to the cache usage threshold, the CPU usage is greater than or equal to the CPU usage threshold, and the memory usage is greater than or equal to the memory usage threshold.

[0018] Optionally, the segment transmission unit also includes: an instruction trigger module, which is used to trigger a destruction instruction when a stream end signal is detected, wherein the destruction instruction is used to close the data stream and release CPU resources and memory resources; a data stream closing module, which is used to close the data stream and recreate a new data stream to transmit the video fragment if a transmission error or transmission timeout occurs during the process of using the data stream to transmit the video fragment.

[0019] Optionally, the video segment transmission device also includes: an upload request unit, which is used to send an upload end request to the server when it is monitored that all the video segments have been transmitted. When the server receives the upload end request, it merges all the video segments and saves the target video.

[0020] Optionally, the video fragment transmission device also includes: a stream identifier allocation unit, which is used to allocate a stream identifier to each data stream after creating multiple data streams based on the QUIC protocol, and the stream identifiers corresponding to each data stream are different.

[0021] According to another aspect of an embodiment of the present invention, a device for receiving video fragments is also provided, which is applied to a server. The device for receiving video fragments includes: a fragment receiving unit, which is used to receive a fragment set transmitted by a video sending end, wherein the fragment set is obtained using any of the above-mentioned video fragment transmission methods; a fragment merging unit, which is used to merge all video fragments in the fragment set according to an integrity priority strategy or a continuity priority strategy, wherein the integrity priority strategy includes: retaining all received video fragments and merging them in ascending order of fragment sequence numbers; the continuity priority strategy includes: dividing all video fragments into fragment groups according to a maximum discontinuity threshold, selecting the fragment group with the largest number, obtaining a fragment sequence number list to be retained based on the fragment sequence number of each video fragment in the selected fragment group, and merging all retained video fragments in ascending order of the sequence numbers in the fragment sequence number list.

[0022] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned video segment transmission methods.

[0023] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the above-mentioned methods for transmitting video fragments.

[0024] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps of any one of the above-mentioned methods for transmitting video segments are implemented.

[0025] In the present invention, the target video is segmented to obtain a segment set including N video segments, and then multiple data streams based on the QUIC protocol are created according to the number of segments, buffer limit requirements and device resources, and the segment set is transmitted to the server using multiple data streams.

[0026] Through the above-mentioned public content, the real-time collected video can be segmented, and multiple data streams based on the QUIC protocol can be flexibly created according to the number of segments, buffer limit requirements and device resources. The created data stream matches the size of the target video to be transmitted, and the created multiple data streams are used to realize segmented transmission and concurrent upload, thereby improving the efficiency of video stream upload, thereby solving the technical problem that the related technology uses a fixed number of data streams to transmit video segments, cannot cope with network fluctuations, and has strong limitations. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0028] Figure 1 is a flow chart of an optional video segment transmission method according to an embodiment of the present invention;

[0029] Figure 2 is a schematic diagram of an optional video slice transmission device according to an embodiment of the present invention;

[0030] Figure 3 The present invention is a hardware structure block diagram of an electronic device (or mobile device) for executing a method for transmitting video segments according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] To facilitate those skilled in the art to understand the present invention, some terms or nouns involved in the embodiments of the present invention are explained below:

[0034] TCP, Transmission Control Protocol, is a connection-oriented, reliable, byte stream-based transport layer communication protocol.

[0035] UDP, User Datagram Protocol, is a connectionless, unreliable network communication protocol. UDP does not establish a connection when transmitting data, nor does it guarantee reliable data transmission, so data packets may be lost or out of order during network transmission.

[0036] QUIC, Quick UDP Internet Connections, is a transport layer protocol based on the UDP protocol, used to quickly and securely transmit data on the network. The QUIC protocol combines the functions of the TCP and TLS protocols to improve the efficiency and security of data transmission by encrypting and compressing data.

[0037] It should be noted that the information (including but not limited to video information, audio information, user personal information, etc.) and data (including but not limited to audio and video data, stored data, displayed data, etc.) collected by the present invention are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data are in compliance with the relevant laws, regulations and standards of the relevant regions, necessary confidentiality measures are taken, and public order and good customs are not violated, and corresponding operation entrances are provided for users to choose to authorize or refuse. For example, an interface is set between the system and the relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or organization.

[0038] It should be noted that in the present invention, when collecting customer information, analyzing customer information, and providing users with corresponding operation entrances, users can choose to agree or reject the automated decision results; if the user chooses to reject, the expert decision process will be entered.

[0039] The QUIC protocol based on the UDP protocol has lower connection establishment delay and better anti-packet loss capability. At the same time, the QUIC protocol supports multiplexing and can transmit multiple data streams at the same time, improving transmission efficiency.

[0040] With the rapid development of mobile Internet, real-time audio and video transmission has become an important application scenario. The following embodiments of the present invention can be applied to various systems / applications / devices for the transmission of video fragments, especially cloud storage video platforms connected to various smart cameras. The present invention proposes a method for uploading video fragments, which realizes efficient fragmented upload of embedded videos based on the multiplexing characteristics of the QUIC protocol, can optimize the video upload process, improve video transmission efficiency, and reduce network latency.

[0041] The present invention is described in detail below in conjunction with various embodiments.

[0042] Embodiment 1

[0043] According to an embodiment of the present invention, an embodiment of a method for transmitting video fragments is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0044] The applicable scenarios of the embodiments of the present invention include, but are not limited to: smart camera cloud storage video uploading scenario, surveillance video uploading scenario, in-vehicle video uploading scenario, and indoor surveillance video uploading scenario.

[0045] Figure 1 is a flowchart of an optional video segment transmission method according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0046] Step S101, fragment the target video to obtain a fragment set including N video fragments, where N is a positive integer.

[0047] Among them, the target video collected in this embodiment can refer to the audio and video information collected by the camera. Each camera is connected to the cloud storage platform through a network connection. The camera acts as a video sender and uploads the collected target video to the server. During the uploading process, the video will be segmented, and then each video segment will be uploaded concurrently through the established data stream. The server receives all the video segments, merges the segments, and obtains the complete video.

[0048] Optionally, step S101 may include: performing fragment processing on the target video according to the single-frame fragmentation requirements to obtain a fragment set. The single-frame fragmentation requirements include: the video fragment starts with an I frame, the video fragment can be parsed and played separately, and the video fragments obtained by the fragmentation processing are assigned with fragment identifiers (i.e., a fragment identifier needs to be assigned to each video fragment, and the fragment identifiers corresponding to each video fragment are different).

[0049] During the fragmentation process, the video sender divides the collected real-time video into multiple video fragments. Each fragment starts with an I frame (an image frame containing a complete picture) and contains one or more I frames to ensure that each fragment can be parsed and played separately, and the loss of a single fragment has no effect on other fragments.

[0050] Among them, the QUIC protocol mentioned in this embodiment adopts multiplexing technology, allowing multiple data streams (defined as streams) to be transmitted in parallel on a single UDP connection. Each data stream has its own sequence number space, which reduces the impact of head-of-line blocking and improves bandwidth utilization and transmission efficiency. Based on this, this embodiment applies the QUIC protocol to the embedded video segment upload scenario.

[0051] Step S102, creating multiple data streams based on the QUIC protocol according to the number of fragments, buffer limit requirements and device resources.

[0052] The buffer limit requirements include: the maximum transmission segment that the buffer is allowed to occupy and the number of transmittable segments; the device resources include: the CPU resources, memory resources, and cache resources that can be used by multiple data streams when transmitting video segments.

[0053] Since different video transmitters are configured with different device resources and different temporary buffers configured for different time periods, in this embodiment, it is necessary to create a data stream based on the total number of video segments, buffer limits, and available resources of the device, and the number of created data streams is determined based on these three factors. Among them, for the buffer, it is necessary to limit the maximum transmission segment that can be occupied at different times, that is, the size of the video segment allowed to be transmitted, and it is also necessary to limit the number of video segments allowed to be transmitted at the same time. This embodiment can efficiently utilize embedded device resources, reduce performance bottlenecks caused by head-of-line blocking, and reduce cache coverage and video data loss, upload interruption, and retransmission caused by network fluctuations.

[0054] According to the above implementation steps, the multiplexing characteristics of the QUIC protocol are adopted in this embodiment to realize the concurrent uploading of target video in segments. In this process, the data stream adapted to the size of the target video to be transmitted and the transmission rate requirements can be flexibly created according to the number of segments, buffer limits and device resources in this embodiment. The number of created data streams can be continuously adjusted to realize the reasonable management of data streams.

[0055] It should be noted that, in this embodiment, after creating multiple data streams based on the QUIC protocol, the video segment transmission method further includes: allocating a stream identifier to each data stream, and the stream identifiers corresponding to each data stream are different. The sender establishes a connection based on the QUIC protocol with the server, and creates multiple streams. The multiple streams are created through the same QUIC connection. Each data stream has a unique stream identifier. For example, a unique stream ID is configured for each data stream, and each data stream is transmitted independently without interfering with each other.

[0056] In this embodiment, each video segment has a unique segment identifier (eg, segment sequence number), each video segment occupies a data stream, and multiple video segments are uploaded concurrently. After receiving the segment, the receiving end records and returns the received segment information, including the segment ID, segment length, and segment integrity.

[0057] Step S103, using multiple data streams to transmit the segment set to the server, wherein each of the multiple data streams uses a time-sharing alternating strategy to transmit the video segment, and the multiple data streams use a concurrent strategy to transmit the video segment.

[0058] It should be noted that the time-sharing alternating strategy mentioned in this embodiment may mean that each data stream allows multiplexing, that is, each data stream can transmit multiple video fragments according to the first-in-first-out principle, each data stream uploads an independent video fragment, and multiple video fragments are independently uploaded / retransmitted without blocking each other, and the same data stream can be reused, reducing the time overhead of creating a connection. The concurrent strategy refers to the use of the multiplexing characteristics of the QUIC protocol to realize concurrent uploading of video fragments for embedded video streams, that is, allowing multiple data streams to be transmitted in fragments at the same time at the same time. The data stream created in this embodiment can be uploaded concurrently, and the retransmission strategy is designed based on the integrity and repeatability of the video fragments to improve the reliability of video stream transmission. Through the above steps, the target video can be fragmented to obtain a fragment set containing N video fragments, and then multiple data streams based on the QUIC protocol are created according to the number of fragments, buffer limit requirements and device resources, and multiple data streams are used to transmit the fragment set to the server, wherein a single data stream uses a time-sharing alternating strategy to transmit video fragments, and multiple data streams use a concurrent strategy to transmit video fragments. In this embodiment, the real-time captured video can be segmented, and multiple data streams based on the QUIC protocol can be flexibly created according to the number of segments, buffer limit requirements, and device resources. The created data stream matches the size of the target video to be transmitted, and the created multiple data streams are used to achieve segmented transmission and concurrent uploading, thereby improving the efficiency of video stream uploading, thereby solving the technical problem that a fixed number of data streams are used to transmit video segments in related technologies, which cannot cope with network fluctuations and has strong limitations.

[0059] In this embodiment, in the process of transmitting video fragments, it is necessary to first perform fragment integrity judgment, and use multiple data streams to transmit the fragment set to the server, including: when the data stream recognizes the fragment end identifier of the video fragment, marking the integrity of the video fragment as true and recording the fragment length; or, when the data stream is abnormally closed or the response transmission operation times out, marking the integrity of the video fragment as no and recording the fragment length.

[0060] For example, if the data stream receives the FIN identifier (i.e., the end of segment identifier), the segment integrity is marked as yes and the segment length is recorded; if the data stream is abnormally closed or the response transmission operation times out, the video segment integrity is marked as no and the segment length is recorded.

[0061] In this embodiment, it is also necessary to perform fragment repeatability judgment in the process of uploading video fragments. Optionally, in the process of transmitting the fragment set to the server using multiple data streams, it also includes: after the data stream receives the fragment upload request from the sender, it is determined that the fragment sequence number in the fragment upload request already exists and the fragment integrity is yes, and the fragment upload request is rejected; or, after the data stream receives the fragment upload request from the sender, it is determined that the fragment sequence number in the fragment upload request already exists, but the fragment integrity is no, then the fragment upload request is received, the video fragment in the fragment upload request is extracted, the fragment length of the extracted video fragment is compared with the fragment length of the fragment with the same fragment sequence number received in the historical process, and the video fragment with the largest fragment length is retained.

[0062] It should be noted that in this embodiment, when using data streams to concurrently upload each video segment, it is necessary to manage data stream resources. Optionally, the video segment transmission method also includes: monitoring the CPU usage, memory usage, the number of streams of all current data streams, and cache usage of the sending end; when the cache usage is lower than the cache usage threshold, the CPU usage is lower than the CPU usage threshold, and the memory usage is lower than the memory usage threshold, creating a new data stream; when the cache usage is greater than or equal to the cache usage threshold, the CPU usage is greater than or equal to the CPU usage threshold, and the memory usage is greater than or equal to the memory usage threshold, stop creating the data stream.

[0063] For example, the sender monitors Cu: CPU usage, Mu: memory usage, Ncur: current number of streams, Bu: cache usage in real time, and sets Nmax: maximum number of streams of the device, Cth: CPU usage threshold, Mth: memory usage threshold, Bth: cache usage threshold based on device specifications. If Bu>Bt and the resource usage of the device does not reach the threshold (that is, Cu is lower than Cth, and Mu is lower than Mth), then add a data stream Stream; when the device resource usage reaches the threshold, block and wait for the existing stream to complete / error / timeout before applying to create a new stream.

[0064] This embodiment can monitor the cache usage rate in real time to feedback the network status, dynamically adjust the number of streams based on easily observable indicator data to adapt to network changes and application requirements, make full use of network bandwidth, improve network transmission performance without requiring a large amount of embedded device computing resources.

[0065] It should be noted that this embodiment also requires dynamic data stream record table management. Specifically, the sender maintains a dynamically updated hash table to record information of all active streams in the current connection, including stream ID, status (such as open, closed, etc.), sent fragment ID and length.

[0066] It should also be noted that when the data transmission on the data stream is completed, the sender will trigger the destruction operation of the data stream, close the data stream, release CPU and memory resources, and delete the information of the data stream from the data stream record table.

[0067] After completing the transmission of all video segments through the data stream, it is necessary to perform a data stream destruction operation. Optionally, the step of using multiple data streams to transmit the segment set to the server also includes: triggering a destruction instruction when a stream end signal is detected, wherein the destruction instruction is used to close the data stream and release CPU resources and memory resources; if a transmission error or transmission timeout occurs during the process of using the data stream to transmit the video segments, the data stream is closed, and a new data stream is created to transmit the video segments.

[0068] For example, this embodiment detects a signal that the stream ends (such as receiving a FIN mark or the stream data is completely transmitted), triggering the data stream destruction process. If an error or timeout occurs during the data transmission process, the sender will actively close the data stream and try to re-create a new data stream for data transmission.

[0069] Furthermore, this embodiment also needs to complete the segment retransmission work, including: if it is detected that the response timeout of the data stream stream is detected, or the segment integrity of the response is negative, then immediately try to recreate a new data stream stream and retransmit the video segment.

[0070] After the sender in this embodiment completes the uploading of all video segments through the data stream, it will inform the server. Optionally, the video segment transmission method also includes: when it is monitored that all video segments have been transmitted, sending an upload end request to the server, wherein the server, upon receiving the upload end request, merges all video segments and saves the target video.

[0071] According to another aspect of an embodiment of the present invention, a method for receiving video segments is further provided, which is applied to a server. The method for receiving video segments includes:

[0072] Receiving a set of segments transmitted by a video transmitter, wherein the set of segments is obtained by using any of the above-mentioned video segment transmission methods;

[0073] According to the completeness priority strategy or the continuity priority strategy, all video fragments in the fragment set are merged, wherein the completeness priority strategy includes: retaining all received video fragments and merging them in ascending order of fragment sequence numbers; the continuity priority strategy includes: dividing all video fragments into fragment groups according to the maximum discontinuity threshold, selecting the fragment group with the largest number, obtaining a fragment sequence number list that needs to be retained based on the fragment sequence number of each video fragment in the selected fragment group, and merging all retained video fragments in ascending order of the sequence numbers in the fragment sequence number list.

[0074] For the continuity priority strategy, you can first choose whether to retain the segments with the integrity of false according to the server configuration, divide all video segments into several groups according to the maximum discontinuity threshold L, select the group with the largest number of segments to obtain the segment sequence number list that needs to be retained (the segment sequence number list is defined as Li, if the number of segments is equal, select the group with a smaller sequence number), and merge all the retained segments in ascending order according to the sequence numbers in Li. For example, the sequence numbers of all received video segments are {1,2,4,16,18,20,30}, L=10, then all video segments are divided into three groups {{1,2,4}, {16,18,20}, {30}}, Li={1,2,4}. The following is a detailed description in conjunction with another embodiment.

[0075] Embodiment 2

[0076] The video segment transmission device provided in this embodiment includes multiple implementation units, each of which corresponds to each implementation step in the above-mentioned embodiment 1. It should be noted that the video segment transmission device mentioned in this embodiment can be applied to the sending end of the cloud storage platform.

[0077] A video slice transmission device provided in this embodiment is applied to a video sending end, that is, a video slice transmission device.

[0078] Figure 2 is a schematic diagram of an optional video slice transmission device according to an embodiment of the present invention, such as Figure 2 As shown, the video slice transmission device may include: a video slice unit 21, a data stream creation unit 22, and a slice transmission unit 23.

[0079] The video segmentation unit 21 is used to segment the target video to obtain a segmentation set including N video segments, where N is a positive integer.

[0080] The data stream creation unit 22 is used to create multiple data streams based on the QUIC protocol according to the number of fragments, buffer limit requirements and device resources, wherein the buffer limit requirements include: the maximum transmission fragments allowed to be occupied by the buffer and the number of transmittable fragments, and the device resources include: CPU resources, memory resources and cache resources that can be used by multiple data streams when transmitting video fragments.

[0081] The segment transmission unit 23 is used to transmit the segment set to the server by using multiple data streams, wherein each of the multiple data streams adopts a time-sharing alternating strategy to transmit the video segment, and the multiple data streams adopt a concurrent strategy to transmit the video segment.

[0082] The above-mentioned video fragment transmission device can be used to fragment the target video through the video fragmentation unit 21 to obtain a fragment set containing N video fragments, and create multiple data streams based on the QUIC protocol according to the number of fragments, buffer limit requirements and device resources through the data stream creation unit 22, and transmit the fragment set to the server through multiple data streams through the fragment transmission unit 23, wherein each of the multiple data streams adopts a time-sharing alternating strategy to transmit video fragments, and multiple data streams adopt a concurrent strategy to transmit video fragments. In this embodiment, the real-time collected video can be fragmented, and multiple data streams based on the QUIC protocol can be flexibly created according to the number of fragments, buffer limit requirements and device resources. The created data stream matches the size of the target video to be transmitted, and the created multiple data streams are used to realize fragment continuation and concurrent uploading, thereby improving the video stream upload efficiency, thereby solving the technical problem that a fixed number of data streams are used to transmit video fragments in the related technology, which cannot cope with network fluctuations and has strong limitations.

[0083] Optionally, the fragment transmission unit includes: a first fragment marking module, which is used to mark the integrity of the video fragment as true and record the fragment length when the data stream recognizes the fragment end mark of the video fragment; or, a second fragment marking module, which is used to mark the integrity of the video fragment as no and record the fragment length when the data stream is abnormally closed or the response transmission operation times out.

[0084] Optionally, the segment transmission unit also includes: a first determination module, which is used to determine that the segment sequence number in the segment upload request already exists and the segment integrity is yes after the data stream receives the segment upload request from the video sending end, and reject the segment upload request; or, a second determination module, which is used to determine that the segment sequence number in the segment upload request already exists but the segment integrity is no after the data stream receives the segment upload request from the video sending end, then receive the segment upload request, extract the video segment in the segment upload request, compare the segment length of the extracted video segment with the segment length of the same segment sequence number received in the historical process, and retain the video segment with the maximum segment length.

[0085] Optionally, the transmission device of the video fragment also includes: a resource monitoring unit, which is used to monitor the CPU usage, memory usage, the number of streams of all current data streams, and the cache usage of the video sending end; a new data stream creation unit, which is used to create a new data stream when the cache usage is lower than the cache usage threshold, the CPU usage is lower than the CPU usage threshold, and the memory usage is lower than the memory usage threshold; and a data stream creation stop unit, which is used to stop creating data streams when the cache usage is greater than or equal to the cache usage threshold, the CPU usage is greater than or equal to the CPU usage threshold, and the memory usage is greater than or equal to the memory usage threshold.

[0086] Optionally, the segment transmission unit also includes: an instruction trigger module, which is used to trigger a destruction instruction when a stream end signal is detected, wherein the destruction instruction is used to close the data stream and release CPU resources and memory resources; a data stream closing module, which is used to close the data stream when a transmission error or transmission timeout occurs during the process of using the data stream to transmit the video segment, and recreate a new data stream to use the new data stream to transmit the video segment.

[0087] Optionally, the video segment transmission device also includes: an upload request unit, which is used to send an upload end request to the server when it is monitored that all video segments have been transmitted. When the server receives the upload end request, it merges all video segments and saves the target video.

[0088] Optionally, the video fragment transmission device also includes: a stream identifier allocation unit, which is used to allocate a stream identifier to each data stream after creating multiple data streams based on the QUIC protocol, and the stream identifiers corresponding to the various data streams are different.

[0089] According to another aspect of an embodiment of the present invention, a device for receiving video fragments is also provided, which is applied to a server. The device for receiving video fragments includes: a fragment receiving unit, which is used to receive a fragment set transmitted by a video sending end, wherein the fragment set is obtained by using any of the above-mentioned video fragment transmission methods; a fragment merging unit, which is used to merge all video fragments in the fragment set according to an integrity priority strategy or a continuity priority strategy, wherein the integrity priority strategy includes: retaining all received video fragments and merging them in ascending order of fragment sequence numbers; the continuity priority strategy includes: dividing all video fragments into fragment groups according to a maximum discontinuity threshold, selecting the fragment group with the largest number, obtaining a fragment sequence number list to be retained based on the fragment sequence number of each video fragment in the selected fragment group, and merging all retained video fragments in ascending order of the sequence numbers in the fragment sequence number list.

[0090] The above-mentioned video segment transmission device may also include a processor and a memory. The above-mentioned video segmentation unit 21, data stream creation unit 22, segment transmission unit 23, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0091] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the multiplexed video fragment upload based on the QUIC protocol can be realized by adjusting the kernel parameters.

[0092] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (fl ash RAM), and the memory includes at least one storage chip.

[0093] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, the computer-readable storage medium including a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the video segment transmission method of any one of the above-mentioned embodiments one.

[0094] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement the video segment transmission method of any one of the above-mentioned embodiments one.

[0095] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the video segment transmission method described in each embodiment of the present application.

[0096] The present application also provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the video segment transmission method described in each embodiment of the present application are implemented.

[0097] Figure 3 1 is a hardware structure block diagram of an electronic device (or mobile device) for executing a video segment transmission method according to an embodiment of the present invention. Figure 3 As shown, the electronic device may include one or more ( Figure 3302a, 302b, ..., 302n are used to illustrate) processor (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 304 for storing data. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 3 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 3 More or fewer components as shown, or with Figure 3 Different configurations shown.

[0098] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0099] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0100] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0101] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0102] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0103] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0104] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for transmitting video segments, characterized in that: Applied to the video sending end, the video fragment transmission method includes: Slice the target video to obtain a slice set including N video slices, where N is a positive integer; Creating multiple data streams based on the QUIC protocol according to the number of fragments, buffer limit requirements, and device resources, wherein the buffer limit requirements include: the maximum transmission fragment allowed to be occupied by the buffer and the number of transmittable fragments, and the device resources include: CPU resources, memory resources, and cache resources that can be used by the multiple data streams when transmitting video fragments; The plurality of data streams are used to transmit the segment set to a server, wherein each of the plurality of data streams adopts a time-sharing alternating strategy to transmit the video segment, and the plurality of data streams adopt a concurrent strategy to transmit the video segment.

2. The video segment transmission method according to claim 1, characterized in that: The adopting the multiple data streams to transmit the shard set to the server includes: In the case where the data stream identifies the end mark of the video segment, marking the integrity of the video segment as true and recording the segment length; or In the case that the data stream is abnormally closed or the response transmission operation times out, the integrity of the video segment is marked as no, and the segment length is recorded.

3. The video segment transmission method according to claim 1, characterized in that: In the process of transmitting the shard set to the server by using the multiple data streams, the process further includes: After the data stream receives the segment upload request from the video transmitter, it is determined that the segment sequence number in the segment upload request already exists and the segment integrity is yes, and the segment upload request is rejected; or, After the data stream receives the segment upload request from the video sender, it is determined that the segment sequence number in the segment upload request already exists, but the segment integrity is no, then the segment upload request is received, the video segments in the segment upload request are extracted, the segment length of the extracted video segments is compared with the segment length of the same segment sequence number received in the historical process, and the video segment with the largest segment length is retained.

4. The video segment transmission method according to claim 1, characterized in that: Also includes: Monitor the CPU usage, memory usage, number of current data streams, and cache usage of the video transmitter; When the cache usage is lower than a cache usage threshold, the CPU usage is lower than a CPU usage threshold, and the memory usage is lower than a memory usage threshold, a new data stream is created; When the cache usage is greater than or equal to the cache usage threshold, the CPU usage is greater than or equal to the CPU usage threshold, and the memory usage is greater than or equal to the memory usage threshold, stop creating the data stream.

5. The video slice transmission method according to claim 1, characterized in that: The adopting the multiple data streams to transmit the shard set to the server also includes: When a stream end signal is detected, a destroy instruction is triggered, wherein the destroy instruction is used to close the data stream and release CPU resources and memory resources; If a transmission error or transmission timeout occurs during the process of using the data stream to transmit the video segment, the data stream is closed, a new data stream is created, and the new data stream is used to transmit the video segment.

6. The video slice transmission method according to claim 1, characterized in that: Also includes: When it is detected that the transmission of all the video segments is completed, an upload end request is sent to the server, wherein the server, upon receiving the upload end request, merges all the video segments and saves the target video.

7. The video slice transmission method according to claim 1, characterized in that: After creating multiple data streams based on the QUIC protocol, it also includes: A flow identifier is allocated to each of the data flows, and the flow identifiers corresponding to the various data flows are different.

8. A method for receiving video fragments, characterized in that: Applied to the server, the method for receiving video segments includes: Receiving a set of slices transmitted by a video sending end, wherein the set of slices is obtained by using the video slice transmission method according to any one of claims 1 to 7; According to the completeness priority strategy or the continuity priority strategy, all video slices in the slice set are merged, wherein the completeness priority strategy includes: retaining all received video slices and merging them in ascending order of slice sequence numbers; the continuity priority strategy includes: dividing all video slices into slice groups according to the maximum discontinuity threshold, selecting the slice group with the largest number, obtaining a slice sequence number list that needs to be retained based on the slice sequence number of each video slice in the selected slice group, and merging all retained video slices in ascending order of the sequence numbers in the slice sequence number list.

9. A video slice transmission device, characterized in that: Applied to the video sending end, the transmission device of the video fragments includes: A video segmentation unit, used for segmenting a target video to obtain a segmentation set including N video segments, where N is a positive integer; A data stream creation unit, configured to create multiple data streams based on the QUIC protocol according to the number of fragments, buffer limit requirements, and device resources, wherein the buffer limit requirements include: the maximum transmission fragment allowed to be occupied by the buffer and the number of transmittable fragments, and the device resources include: CPU resources, memory resources, and cache resources that can be used by the multiple data streams when transmitting video fragments; The segment transmission unit is used to transmit the segment set to the server by using the multiple data streams, wherein each of the multiple data streams adopts a time-sharing alternating strategy to transmit the video segment, and the multiple data streams adopt a concurrent strategy to transmit the video segment.

10. A device for receiving video slices, characterized in that: Applied to the server, the receiving device of the video fragments includes: A slice receiving unit, configured to receive a slice set transmitted by a video sending end, wherein the slice set is obtained by using the video slice transmission method according to any one of claims 1 to 7; The fragment merging unit is used to merge all video fragments in the fragment set according to the completeness priority strategy or the continuity priority strategy, wherein the completeness priority strategy includes: retaining all received video fragments and merging them in ascending order of fragment sequence numbers; the continuity priority strategy includes: dividing all video fragments into various fragment groups according to a maximum discontinuity threshold, selecting the fragment group with the largest number, obtaining a fragment sequence number list that needs to be retained based on the fragment sequence number of each video fragment in the selected fragment group, and merging all retained video fragments in ascending order of the sequence numbers in the fragment sequence number list.

11. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the video segment transmission method described in any one of claims 1 to 7.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the video segment transmission method according to any one of claims 1 to 7 are implemented.

Images