Video stream processing method and device, computer equipment and storage medium

Abstract

The invention relates to a video stream processing method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: receiving a playing request of a recorded video, and determining storage addresses respectively corresponding to a plurality of video fragments to be played and a playing sequence of the plurality of video fragments based on the playing request; the method comprises the following steps: acquiring a current video fragment in sequence, downloading the current video fragment from a cloud storage address corresponding to the current video fragment in a segmented manner through a plurality of threads in a thread pool based on a video format of the current video fragment, and respectively storing a plurality of obtained sub-fragments in corresponding positions of a cache queue; and obtaining each sub-fragment from the cache queue in sequence, and generating a playing address of the recorded video based on each sub-fragment. According to the invention, the video stream playing continuity can be improved.

Description

Technical Field

[0001] The present application relates to the field of video transmission technology, and in particular to a video stream processing method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] With the development of 5G networks and video technology, video recording has penetrated into all aspects of people's lives. In many scenarios, such as security monitoring, entertainment and education, transportation and medical care, real-time viewing of video streams is required.

[0003] Video streaming is a technology that converts recorded video content into a streaming media format suitable for transmission on the network. It breaks the recorded video into multiple data packets and transmits these data packets in real time on the network to achieve the function of downloading and playing at the same time. However, in the related video streaming processing, for multiple video clips, each video clip can only be viewed one by one, which affects the continuity of video playback. Summary of the invention

[0004] Based on this, it is necessary to provide a video stream processing method, device, computer equipment, computer-readable storage medium and computer program product that can improve the continuity of recorded video playback in response to the above technical problems.

[0005] In a first aspect, the present application provides a video stream processing method. The method comprises:

[0006] receiving a playback request for a recorded video, and determining storage addresses corresponding to a plurality of recorded video segments to be played and a playback order of the plurality of recorded video segments based on the playback request;

[0007] Obtaining the current video clip in sequence, and based on the video format of the current video clip, downloading the current video clip in segments from the cloud storage address corresponding to the current video clip through multiple threads in the thread pool, and storing the obtained multiple sub-segments in corresponding positions of the cache queue respectively;

[0008] The sub-segments are sequentially obtained from the cache queue, and based on the sub-segments, a playback address of the recorded video is generated.

[0009] In one embodiment, the receiving a request to play a recorded video and determining storage addresses corresponding to a plurality of recorded segments to be played and a play order of the plurality of recorded segments based on the play request include:

[0010] Parsing the playback request to obtain a storage address list corresponding to the recorded video and a desired playback time range;

[0011] Sort the storage addresses in the storage address list based on the start time and end time of each storage address in the storage address list;

[0012] Based on the expected playback time range, the storage addresses corresponding to the sorted video clips are screened to obtain the storage addresses corresponding to the multiple video clips to be played and the playback order of the multiple video clips.

[0013] In one embodiment, the filtering of storage addresses corresponding to the sorted video clips based on the expected playback time range includes:

[0014] Based on the start time of the expected playback time range, determine the storage address of the first video segment to be played from the storage addresses corresponding to the sorted video segments; and based on the storage address of the first video segment to be played, obtain the first video segment to be played;

[0015] Determining a key video frame from the first video segment to be played based on the start time of the first video segment to be played and the start time of the expected playback time range;

[0016] The key video frame is used as the starting frame of multiple video clips, and based on the end time of the expected playback time range, subsequent video clips are screened from the video clips after the first video clip to be played to obtain multiple video clips and corresponding storage addresses.

[0017] In one of the embodiments, based on the video format of the current video clip, downloading the current video clip in segments from a storage address corresponding to the current video clip through multiple threads in a thread pool includes:

[0018] In a case where the video format of the current video clip is a program stream format, determining the byte ranges corresponding to the sub-segments based on the total size of the current video clip and the size of each sub-segment;

[0019] Through multiple threads in the thread pool, data acquisition requests are respectively sent to the storage address corresponding to the current video segment to obtain each sub-segment; wherein the data acquisition request carries the byte range of the corresponding sub-segment.

[0020] In one of the embodiments, based on the video format of the current video clip, downloading the current video clip in segments from a storage address corresponding to the current video clip through multiple threads in a thread pool includes:

[0021] When the video format of the current video clip is the HLS streaming media format, downloading the index file from the storage address corresponding to the current video clip through the target thread in the thread pool;

[0022] Based on the index address of each sub-file in the index file, the sub-segments corresponding to the index address are downloaded respectively through other threads in the thread pool.

[0023] In one embodiment, sequentially acquiring each of the sub-segments from the cache queue and generating a playback address of the recorded video based on each of the sub-segments includes:

[0024] Sequentially acquiring each of the sub-segments from the cache queue, and performing video decoding on each of the sub-segments according to a video decoding method corresponding to each of the sub-segments to obtain frame data;

[0025] The frame data is encapsulated into a recorded video in a target format, and a playback address of the recorded video is generated; wherein the target format includes at least one or more of the following: rtmp format, rtsp format, hls format, and webrtc format.

[0026] In a second aspect, the present application also provides a video stream processing device. The device comprises:

[0027] A receiving module, configured to receive a playback request for a recorded video, and determine storage addresses corresponding to a plurality of recorded video segments to be played and a playback order of the plurality of recorded video segments based on the playback request;

[0028] A download module, used to sequentially obtain the current video clip, and based on the video format of the current video clip, download the current video clip in segments from the cloud storage address corresponding to the current video clip through multiple threads in the thread pool, and store the obtained multiple sub-segments in corresponding positions of the cache queue respectively;

[0029] The generating module is used to sequentially obtain each of the sub-segments from the cache queue, and generate a playback address of the recorded video based on each of the sub-segments.

[0030] In one embodiment, the receiving module is further used for:

[0031] Parsing the playback request to obtain a storage address list corresponding to the recorded video and a desired playback time range;

[0032] Sort the storage addresses in the storage address list based on the start time and end time of each storage address in the storage address list;

[0033] Based on the expected playback time range, the storage addresses corresponding to the sorted video clips are screened to obtain the storage addresses corresponding to the multiple video clips to be played and the playback order of the multiple video clips.

[0034] In one embodiment, the receiving module is further used for:

[0035] Based on the start time of the expected playback time range, determine the storage address of the first video segment to be played from the storage addresses corresponding to the sorted video segments; and based on the storage address of the first video segment to be played, obtain the first video segment to be played;

[0036] Determining a key video frame from the first video segment to be played based on the start time of the first video segment to be played and the start time of the expected playback time range;

[0037] The key video frame is used as the starting frame of multiple video clips, and based on the end time of the expected playback time range, subsequent video clips are screened from the video clips after the first video clip to be played to obtain multiple video clips and corresponding storage addresses.

[0038] In one embodiment, the download module is further used to:

[0039] In a case where the video format of the current video clip is a program stream format, determining the byte ranges corresponding to the sub-segments based on the total size of the current video clip and the size of each sub-segment;

[0040] Through multiple threads in the thread pool, data acquisition requests are respectively sent to the storage address corresponding to the current video segment to obtain each sub-segment; wherein the data acquisition request carries the byte range of the corresponding sub-segment.

[0041] In one embodiment, the download module is further used to:

[0042] When the video format of the current video clip is the HLS streaming media format, downloading the index file from the storage address corresponding to the current video clip through the target thread in the thread pool;

[0043] Based on the index address of each sub-file in the index file, the sub-segments corresponding to the index address are downloaded respectively through other threads in the thread pool.

[0044] In one embodiment, the generating module is further used for:

[0045] Sequentially acquiring each of the sub-segments from the cache queue, and performing video decoding on each of the sub-segments according to a video decoding method corresponding to each of the sub-segments to obtain frame data;

[0046] The frame data is encapsulated into a recorded video in a target format, and a playback address of the recorded video is generated; wherein the target format includes at least one or more of the following: rtmp format, rtsp format, hls format, and webrtc format.

[0047] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, an embodiment of the method described in any one of the embodiments of the present disclosure is implemented.

[0048] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, an embodiment of the method described in any one of the embodiments of the present disclosure is implemented.

[0049] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor, an embodiment of the method described in any one of the embodiments of the present disclosure is implemented.

[0050] By obtaining storage addresses corresponding to multiple video clips from a playback request, determining the playback order of multiple video clips, and based on the video format of the current video clip, downloading the current video clip in segments from the cloud storage address corresponding to the current video clip through multiple threads in a thread pool, and storing the obtained multiple sub-segments in corresponding positions of a cache queue respectively; obtaining each of the sub-segments from the cache queue in turn, and generating the playback address of the video based on each of the sub-segments. The effect of continuous playback between different video clips can be achieved. Compared with the prior art, the user manually clicks on each video clip to play the video clip, and then clicks on another video clip to play another video clip, which is not only cumbersome to operate, but also interrupts between the two video clips. In addition, for video clips of different video formats, the embodiment of the present disclosure does not need to manually switch protocols, and the embodiment of the present disclosure can achieve the effect of automatic splicing and continuous playback of video clips of different formats. In addition, the setting of the thread pool and the cache queue ensures the real-time performance of the video clips. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A diagram showing an application environment of a video stream processing method in an embodiment;

[0052] Figure 2A schematic diagram of a video stream processing method in one embodiment;

[0053] Figure 3 is a flow chart of a video stream processing method in another embodiment;

[0054] Figure 4 is a flow chart of a video stream processing method in another embodiment;

[0055] Figure 5 A system architecture diagram of a video stream processing method in one embodiment;

[0056] Figure 6 is a structural block diagram of a video stream processing device in one embodiment;

[0057] Figure 7 is an internal structure diagram of a computer device in one embodiment; DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0059] The video stream processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the first terminal 102 communicates with the server 104 through the network, and the second terminal 106 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other servers. The second terminal 106 stores the recorded video clips on the server or cloud server, and the server 104 processes the video clips, and the processed video is used for transmission or playback on the first terminal 102. Among them, the first terminal 102 can be but not limited to various desktop computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The second terminal 106 can be but not limited to smart phones, cameras, portable wearable devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0060] In one embodiment, Figure 2 As shown, a video streaming method is provided, which is applied to Figure 1 Taking the server 104 in the example as an example, the following steps are included:

[0061] Step S201: receiving a playback request for a recorded video, and determining storage addresses corresponding to a plurality of recorded segments to be played and a playback order of the plurality of recorded segments based on the playback request.

[0062] Specifically, on the first terminal, the user can upload the storage address of one or more video clips, and these storage addresses are carried in the play request and sent to the server. Optionally, the formats of the multiple video clips can be the same video format or different video formats. In addition, on the first terminal, a control for stopping playback and a control for controlling playback progress or playback rate can also be set, and a request to stop playback and a request to control playback are generated accordingly.

[0063] In an exemplary embodiment, the start time and end time of each video segment can be obtained, and the playback order of multiple video segments can be determined according to the time sequence. For example, the start time of video segment 1 is t1, the start time of route segment 2 is t2, and the start time of video segment 3 is t3. If t1 is less than t2 and less than t3, the playback order of the above video segments is t1, t2, and t3.

[0064] Step S203, sequentially obtain the current video clips, and based on the video format of the current video clips, download the current video clips in segments from the cloud storage address corresponding to the current video clips through multiple threads in the thread pool, and store the obtained multiple sub-segments in corresponding positions of the cache queue respectively.

[0065] Specifically, each video clip can be obtained according to the playback order of the video clips. When the current video clip is obtained, the download method of the corresponding video clip is determined according to the video format of the current video clip. In an exemplary embodiment, a thread pool and a cache queue are created to download the current video clip in segments and store the downloaded sub-segments. For example, the current video clip is divided into several sub-segments, each thread downloads the corresponding sub-segment, stores the downloaded sub-segments in the cache queue, and sets a number for the sub-segment.

[0066] Step S205 , obtaining each of the sub-segments from the cache queue in sequence, and generating a playback address of the recorded video based on each of the sub-segments.

[0067] In the disclosed embodiment, each sub-segment can be obtained in the order of the number in the cache queue. In an exemplary embodiment, for example, the video playback format of the first terminal is the same as the original format of each sub-segment, and each sub-segment can be spliced ​​to obtain a spliced ​​video, and a stream address of the spliced ​​video is generated. The first terminal pulls the address and can play the spliced ​​video in real time. In another exemplary embodiment, for example, the video format of the first terminal is different from the original format of each sub-segment, and each sub-segment can be decoded according to the corresponding video format to obtain video frame data. After the video frame data is sorted according to the timestamp, the sorted video frame data is encoded into a video in a target format, and a playback address of the video in the target format is generated. Optionally, the spliced ​​video or the video in the target format can come from the same video clip or from different video clips.

[0068] In the above embodiment, the storage addresses corresponding to the multiple video clips are obtained from the playback request to determine the playback order of the multiple video clips. Based on the video format of the current video clip, the current video clip is downloaded from the cloud storage address corresponding to the current video clip by multiple threads in the thread pool, and the obtained multiple sub-segments are respectively stored in the corresponding positions of the cache queue; each of the sub-segments is obtained from the cache queue in turn, and the playback address of the video is generated based on each of the sub-segments. The effect of continuous playback between different video clips can be achieved. Compared with the prior art, the user manually clicks on each video clip to play the video clip, and then clicks on another video clip to play another video clip. Not only is the operation cumbersome, but there will be an interruption between the two video clips. In addition, for video clips of different video formats, the embodiment of the present disclosure does not need to manually switch the protocol, and the embodiment of the present disclosure can achieve the effect of automatic splicing and continuous playback of video clips of different formats. In addition, the setting of the thread pool and the cache queue ensures the real-time performance of the video clips.

[0069] In one embodiment, reference Figure 3 As shown, the receiving a playback request of a recorded video, and determining storage addresses corresponding to a plurality of recorded segments to be played and a playback order of the plurality of recorded segments based on the playback request, includes:

[0070] Step S301, parsing the playback request to obtain a storage address list corresponding to the recorded video and a desired playback time range.

[0071] Specifically, on the first terminal, the user can upload the storage addresses of one or more video clips, which are carried in the play request in the form of a list and sent to the server. The desired play time range represents the specific time period of the video clip that the user wants to watch, such as 8 am to 8:30 am.

[0072] Step S303: sorting the storage addresses in the storage address list based on the start time and end time of each storage address in the storage address list.

[0073] Similar to the above embodiment, the start time and end time of each video segment can be obtained, and the playback order of multiple video segments can be determined according to the time sequence. For example, the start time of video segment 1 is t1, the start time of route segment 2 is t2, and the start time of video segment 3 is t3. If t1 is less than t2 and less than t3, then the playback order of the above video segments is t1, t2, and t3.

[0074] Step S305 , based on the expected playback time range, the storage addresses corresponding to the sorted video clips are screened to obtain the storage addresses corresponding to the multiple video clips to be played and the playback order of the multiple video clips.

[0075] In an exemplary embodiment, a video clip whose start time and end time are within the desired playback time range or a video clip whose start time and end time overlap with the desired playback time range can be selected to obtain storage addresses corresponding to the multiple video clips to be played and the playback order of the video clips.

[0076] In the above embodiment, by obtaining the desired playback time range and filtering the storage address list corresponding to the selected recorded video, the recorded video clips that meet the user's desired playback can be filtered out, thereby realizing the selection of the desired playback time range of multiple recorded video clips.

[0077] In one embodiment, the filtering of storage addresses corresponding to the sorted video clips based on the expected playback time range includes:

[0078] Step S301, based on the start time of the expected playback time range, determine the storage address of the first video segment to be played from the storage addresses corresponding to the sorted video segments; and based on the storage address of the first video segment to be played, obtain the first video segment to be played.

[0079] In an exemplary embodiment, a video segment that passes or overlaps with the start time of the desired playback time range can be selected from the sorted video segments as the storage address of the first video segment to be played. For example, the start time of the desired playback time range is 12:02:00, and the start time and end time of video segment A are 12:00:00 and 13:00:00 respectively. Then video segment A is selected as the first video segment to be played.

[0080] Step S303: determining a key video frame from the first video segment to be played based on the start time of the first video segment to be played and the start time of the expected playback time range.

[0081] In the above embodiment, for example, the start time of the first video segment A to be played is 12:00:00, and the start time of the desired playback time range is 12:02:00, then a key frame is selected from 12:00:00 to 12:02:00 of the video segment A as a key video frame, wherein the key frame may include independently coded video frames, such as I frames or IDR frames.

[0082] Step S305, taking the key video frame as the start frame of multiple video segments, based on the end time of the desired playback time range, selecting subsequent video segments from the video segments after the first video segment to be played, to obtain multiple video segments and corresponding storage addresses.

[0083] In an exemplary embodiment, the key video frame is used as the start frame of multiple video clips. Further, for example, if the end time of the expected playback time range is 18:00:00, the video clips whose end time is before 18:00:00 are screened from the video clips after the first video clip to be played. For example, the end time of video clip B is 15:00:00, the end time of video clip C is 17:00:00, the start time of video clip D is 17:50:00 and the end time is 18:20:00, then the end time of video clip B and the end time of video clip C are both before the expected end time 18:00:00, and video clip B and video clip C can be used as subsequent video clips. Since the expected end time is within the start time and end time range of video clip D, video clip D can also be used as a subsequent video clip.

[0084] In the disclosed embodiment, a key video frame is determined from the first video clip to be played based on the start time of the first video clip to be played and the start time of the expected playback time range. If there are multiple key video frames, the one closest to the start time of the expected playback time range is selected. Compared with the prior art of first pulling the video clip from the cloud and then streaming it, there is a delay, which causes the user to wait too long and reduces the viewing experience. In the disclosed embodiment, the key video frame is directly determined from the first video clip to be played, which can increase the screen opening speed of the first terminal and ensure the accuracy, timeliness and integrity of the user's playback image.

[0085] In one embodiment, based on the video format of the current video segment, downloading the current video segment in segments from the storage address corresponding to the current video segment through multiple threads in a thread pool includes:

[0086] When the video format of the current video clip is a program stream format, determining the byte ranges corresponding to the sub-segments based on the total size of the current video clip and the size of each sub-segment;

[0087] Through multiple threads in the thread pool, data acquisition requests are respectively sent to the storage address corresponding to the current video segment to obtain each sub-segment; wherein the data acquisition request carries the byte range of the corresponding sub-segment.

[0088] Among them, the program stream format (PS) is a streaming media format that combines one or more PES packets with a common time base into a single stream.

[0089] Specifically, for example, the total size of the current video clip is 200Mb, and the storage location corresponding to the current route clip is divided into units of 4Mb as the size of a sub-segment, for example, sub-segment A corresponds to 0-4194304, and sub-segment B corresponds to 4194305-8388609. In an exemplary embodiment, threads in a thread pool are used to obtain each sub-segment respectively, for example, thread 1 sends data acquisition requests (such as http requests) of 0-4194304 to the cloud or server to obtain sub-segment A; thread 2 sends data acquisition requests (such as http requests) of 4194305-8388609 to the cloud or server to obtain sub-segment B.

[0090] In the above embodiment, by dividing the current video clip into multiple sub-segments and using threads in the thread pool to respectively obtain corresponding sub-segments, the efficiency of data downloading can be improved and the real-time performance of video playback can be ensured.

[0091] In one embodiment, based on the video format of the current video segment, downloading the current video segment in segments from the storage address corresponding to the current video segment through multiple threads in a thread pool includes:

[0092] When the video format of the current video clip is the HLS streaming media format, downloading the index file from the storage address corresponding to the current video clip through the target thread in the thread pool;

[0093] Based on the index address of each sub-file in the index file, the sub-segments corresponding to the index address are downloaded respectively through other threads in the thread pool.

[0094] Among them, the HLS streaming format is an adaptive bitrate streaming media transmission protocol based on HTTP. It is mainly used to realize the transmission and playback of streaming media content. It can be a video recording format or a real-time playback protocol.

[0095] In an exemplary embodiment, a thread can be selected from the thread pool as a target thread, and the target thread is used to download the index file of the current video clip, such as an m3u8 file. The index file contains the index address of each sub-file, and other threads in the thread pool are used to download the sub-files (ts files) corresponding to each index address. The sub-files can be used as sub-segments of the current video clip.

[0096] In the above embodiment, the threads in the thread pool download the sub-files corresponding to the current video clip respectively, which can improve the efficiency of data downloading and ensure the real-time performance of video playback.

[0097] In one embodiment, sequentially acquiring each of the sub-segments from the cache queue and generating a playback address of the recorded video based on each of the sub-segments includes:

[0098] The sub-segments are sequentially acquired from the cache queue, and video decoding is performed on the sub-segments according to the video decoding mode corresponding to the sub-segments to obtain frame data.

[0099] The frame data is encapsulated into a recorded video in a target format, and a playback address of the recorded video is generated; wherein the target format includes at least one or more of the following: rtmp format, rtsp format, hls format, and webrtc format.

[0100] In the disclosed embodiment, each sub-segment can be obtained in sequence according to the number of the cache queue. In an exemplary embodiment, decoders of multiple protocols can be set to decode the sub-band based on the format of the sub-segment. Optionally, the decoded frame data can be cached and sorted according to the timestamp after audio and video synchronization.

[0101] In an exemplary embodiment, the frame data in the buffer can be reorganized and packaged to generate a recorded video in a target format, wherein the target format can include at least one or more of rtmp format, rtsp format, hls format, and webrtc format.

[0102] In the above embodiment, each sub-segment is decoded and encapsulated into a recorded video in a target format, which meets the requirement of playing and forwarding the recorded video in any format.

[0103] In a specific embodiment, reference Figure 4As shown, the present application provides a video stream processing method, which includes: a video segment acquisition stage, a sub-segment download stage, a sub-segment decoding stage, and a frame data encoding stage.

[0104] In the video clip acquisition stage, specifically, a cloud video address list is obtained from the playback request of the video, the video clips are sorted according to the start time and end time of the video clips, and the sorted video clips are filtered according to the time range that the user expects to play.

[0105] In the sub-segment downloading stage, the sorted video segments are downloaded in sequence, and the storage address corresponding to the first video segment is taken as an example to determine the type of the video segment. When the video format of the current video segment is the program stream format (PS), the byte range corresponding to each sub-segment is determined based on the total size of the current video segment and the size of each sub-segment; through multiple threads in the thread pool, data acquisition requests are sent to the cloud storage address corresponding to the current video segment to obtain each sub-segment; wherein the data acquisition request carries the byte range of the corresponding sub-segment. When the video format of the current video segment is the HLS streaming media format, the index file is downloaded from the storage address corresponding to the current video segment through the target thread in the thread pool; based on the index address of each sub-file in the index file, the sub-segments corresponding to the index address are downloaded through other threads in the thread pool.

[0106] In the sub-segment decoding stage, each of the sub-segments is sequentially obtained from the cache queue, and video decoding is performed on each of the sub-segments according to a video decoding method corresponding to each of the sub-segments to obtain frame data.

[0107] In the frame data encoding stage, the frame data is encapsulated into a video of a target format, and a playback address of the video is generated; wherein the target format includes at least one or more of the following: rtmp format, rtsp format, hls format, webrtc format. The playback address can be played directly or streamed.

[0108] Determine whether the user-specified end time has been reached, whether the remaining data in the cache meets the preset playback volume, and whether the download of all video lists has been completed. If so, end it, otherwise obtain the next video clip. Optionally, a timeout mechanism can be set. For example, in the process of video stream processing, corresponding timeout times and timeout corresponding policies are set for cloud video pulling, stream parsing and decoding, and stream forwarding. When the corresponding processing cannot be completed within the specified time due to network reasons or abnormal stream data, timeout exception handling can ensure that the system will not wait indefinitely, thereby avoiding waste of system resources and potential system crashes. At the same time, it can quickly provide feedback to the user after the operation times out, and the user can perform corresponding operations based on the feedback, such as retrying, canceling the request, or selecting other operations, thereby improving the user experience.

[0109] In one embodiment, reference Figure 5As shown in the figure, the video architecture of video stream processing, wherein the user request API module provides the user with an API interface, mainly including the start splicing streaming request, the stop splicing streaming request, the control splicing streaming request, etc. Among them, in the start splicing streaming interface, the user can pass in the http address of one or more cloud video clips, which can be in the same packaging format or in multiple packaging formats such as ps / hls. And indicate the start time and end time of these clips, as well as the time range you want to play. In the control streaming interface, the user can specify the operation type through specific control parameters, including fast forward, rewind, pause, resume, double speed and other operations. In the stop interface, the user can stop the streaming operation. The cloud video pull module, this module is the core module of splicing streaming, realizes the cloud video pull and cache, and transmits it to the stream parsing module for decoding. In order to improve the efficiency of cloud video pull, we use multi-threaded http segmentation to download, create a thread pool and a variable-length cache queue to manage the concurrency of multiple download threads and the cache of downloaded content. A complete video clip is cut into several small pieces. Each thread will be assigned a small piece of the video clip, and the downloaded data will be input into the corresponding number in the cache queue. At the same time, the head of the cache queue will be continuously input into the stream parsing module. The multi-protocol stream parsing module, the stream transmitted through the cloud video pull module may have multiple encapsulation formats. The stream parsing module provides multiple decoders, which can decode streams of different protocols and generate frame data. When it is recognized that the video cache queue meets the parsing requirements, the data will be pulled in order, and then the corresponding decoder will be selected according to the data format. For the parsed frame data, the cache queue is used for storage, audio and video synchronization is performed, and then sorted according to the timestamp, and finally transmitted to the stream forwarding module. The multi-protocol stream forwarding module reorganizes and encapsulates the frame data output by the stream parsing module, generates stream addresses of multiple protocols such as rtmp / rtsp / hls / webrtc, and users can play the streamed video in real time by pulling this address. At the same time, if the user provides a data forwarding address, the frame data can also be encapsulated as an rtp data packet and forwarded to the user's destination address. Timeout exception handling module, this module runs throughout the life cycle of splicing streaming to improve system reliability and optimize user experience. Corresponding timeouts and timeout corresponding strategies are set for cloud video pulling, stream parsing and decoding, and stream forwarding. When the corresponding processing cannot be completed within the specified time due to network reasons or abnormal stream data, timeout exception handling can ensure that the system will not wait indefinitely, thereby avoiding waste of system resources and potential system crashes. At the same time, it can quickly provide feedback to the user after the operation times out, and the user can perform corresponding operations based on the feedback, such as retrying, canceling the request, or selecting other operations, thereby improving the user experience.

[0110] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0111] Based on the same inventive concept, the embodiment of the present application also provides a video stream processing device for implementing the video stream processing method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in the one or more video stream processing device embodiments provided below can refer to the limitations of the video stream processing method above, and will not be repeated here.

[0112] In one embodiment, Figure 6 As shown, a video stream processing device is provided, comprising:

[0113] The receiving module 601 is used to receive a playback request of a recorded video, and determine storage addresses corresponding to a plurality of recorded video segments to be played and a playback order of the plurality of recorded video segments based on the playback request;

[0114] The download module 603 is used to sequentially obtain the current video clips, and based on the video format of the current video clips, download the current video clips in segments from the cloud storage address corresponding to the current video clips through multiple threads in the thread pool, and store the obtained multiple sub-segments in corresponding positions of the cache queue respectively;

[0115] The generating module 605 is used to sequentially obtain each of the sub-segments from the cache queue, and generate a playback address of the recorded video based on each of the sub-segments.

[0116] In one embodiment, the receiving module is further used for:

[0117] Parsing the playback request to obtain a storage address list corresponding to the recorded video and a desired playback time range;

[0118] Sort the storage addresses in the storage address list based on the start time and end time of each storage address in the storage address list;

[0119] Based on the expected playback time range, the storage addresses corresponding to the sorted video clips are screened to obtain the storage addresses corresponding to the multiple video clips to be played and the playback order of the multiple video clips.

[0120] In one embodiment, the receiving module is further used for:

[0121] Based on the start time of the expected playback time range, determine the storage address of the first video segment to be played from the storage addresses corresponding to the sorted video segments; and based on the storage address of the first video segment to be played, obtain the first video segment to be played;

[0122] Determining a key video frame from the first video segment to be played based on the start time of the first video segment to be played and the start time of the expected playback time range;

[0123] The key video frame is used as the starting frame of multiple video clips, and based on the end time of the expected playback time range, subsequent video clips are screened from the video clips after the first video clip to be played to obtain multiple video clips and corresponding storage addresses.

[0124] In one embodiment, the download module is further used to:

[0125] In a case where the video format of the current video clip is a program stream format, determining the byte ranges corresponding to the sub-segments based on the total size of the current video clip and the size of each sub-segment;

[0126] Through multiple threads in the thread pool, data acquisition requests are respectively sent to the storage address corresponding to the current video segment to obtain each sub-segment; wherein the data acquisition request carries the byte range of the corresponding sub-segment.

[0127] In one embodiment, the download module is further used to:

[0128] When the video format of the current video clip is the HLS streaming media format, downloading the index file from the storage address corresponding to the current video clip through the target thread in the thread pool;

[0129] Based on the index address of each sub-file in the index file, the sub-segments corresponding to the index address are downloaded respectively through other threads in the thread pool.

[0130] In one embodiment, the generating module is further used for:

[0131] Sequentially acquiring each of the sub-segments from the cache queue, and performing video decoding on each of the sub-segments according to a video decoding method corresponding to each of the sub-segments to obtain frame data;

[0132] The frame data is encapsulated into a recorded video in a target format, and a playback address of the recorded video is generated; wherein the target format includes at least one or more of the following: rtmp format, rtsp format, hls format, and webrtc format.

[0133] Each module in the above video stream processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0134] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store video stream processing data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a video stream processing method is implemented.

[0135] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0136] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0137] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0138] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0139] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A video stream processing method, characterized in that: Applied to a server, the method comprises: receiving a playback request for a recorded video, and determining storage addresses corresponding to a plurality of recorded video segments to be played and a playback order of the plurality of recorded video segments based on the playback request; Obtaining the current video clip in sequence, and based on the video format of the current video clip, downloading the current video clip in segments from the cloud storage address corresponding to the current video clip through multiple threads in the thread pool, and storing the obtained multiple sub-segments in corresponding positions of the cache queue respectively; The sub-segments are sequentially obtained from the cache queue, and based on the sub-segments, a playback address of the recorded video is generated.

2. The method according to claim 1, characterized in that The receiving a playback request of a recorded video, and determining storage addresses corresponding to a plurality of recorded segments to be played and a playback order of the plurality of recorded segments based on the playback request, includes: Parsing the playback request to obtain a storage address list corresponding to the recorded video and a desired playback time range; Sort the storage addresses in the storage address list based on the start time and end time of each storage address in the storage address list; Based on the expected playback time range, the storage addresses corresponding to the sorted video clips are screened to obtain the storage addresses corresponding to the multiple video clips to be played and the playback order of the multiple video clips.

3. The method according to claim 2, characterized in that The step of screening the storage addresses corresponding to the sorted video clips based on the expected playback time range includes: Based on the start time of the expected playback time range, determine the storage address of the first video segment to be played from the storage addresses corresponding to the sorted video segments; and based on the storage address of the first video segment to be played, obtain the first video segment to be played; Determining a key video frame from the first video segment to be played based on the start time of the first video segment to be played and the start time of the expected playback time range; The key video frame is used as the starting frame of multiple video clips, and based on the end time of the expected playback time range, subsequent video clips are screened from the video clips after the first video clip to be played to obtain multiple video clips and corresponding storage addresses.

4. The method according to claim 1, characterized in that Based on the video format of the current video clip, downloading the current video clip in segments from a storage address corresponding to the current video clip through multiple threads in a thread pool includes: When the video format of the current video clip is a program stream format, determining the byte ranges corresponding to the sub-segments based on the total size of the current video clip and the size of each sub-segment; Through multiple threads in the thread pool, data acquisition requests are respectively sent to the storage address corresponding to the current video segment to obtain each sub-segment; wherein the data acquisition request carries the byte range of the corresponding sub-segment.

5. The method according to claim 1, characterized in that Based on the video format of the current video clip, downloading the current video clip in segments from a storage address corresponding to the current video clip through multiple threads in a thread pool includes: When the video format of the current video clip is the HLS streaming media format, downloading the index file from the storage address corresponding to the current video clip through the target thread in the thread pool; Based on the index address of each sub-file in the index file, the sub-segments corresponding to the index address are downloaded respectively through other threads in the thread pool.

6. The method according to claim 1, characterized in that The obtaining each of the sub-segments from the cache queue in sequence and generating a playback address of the recorded video based on each of the sub-segments includes: Obtaining each of the sub-segments from the cache queue in sequence, and performing video decoding on each of the sub-segments according to a video decoding method corresponding to each of the sub-segments to obtain frame data; The frame data is encapsulated into a recorded video in a target format, and a playback address of the recorded video is generated; wherein the target format includes at least one or more of the following: rtmp format, rtsp format, hls format, and webrtc format.

7. A video stream processing device, characterized in that: The device comprises: A receiving module, configured to receive a playback request for a recorded video, and determine storage addresses corresponding to a plurality of recorded video segments to be played and a playback order of the plurality of recorded video segments based on the playback request; A download module, used to sequentially obtain the current video clip, and based on the video format of the current video clip, download the current video clip in segments from the cloud storage address corresponding to the current video clip through multiple threads in the thread pool, and store the obtained multiple sub-segments in corresponding positions of the cache queue respectively; The generating module is used to sequentially obtain each of the sub-segments from the cache queue, and generate a playback address of the recorded video based on each of the sub-segments.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images