A high-performance video data distribution method

Abstract

The application provides a high-performance video data distribution method, comprising the following steps: arranging multiple threads on multiple CPU cores of a transcoding server; according to video playing requests of multiple user clients, distributing the video playing requests of the multiple user clients to the multiple threads of the transcoding server, so that each thread distributes video transmitted by a camera to a corresponding client browser according to the video playing request of the user client allocated to the thread. The application arranges all threads on multiple CPU cores of a transcoding server, and the threads in the multiple CPU cores can distribute video data to multiple client browsers in parallel, so that the number of thread switching is reduced, while the data can be distributed by using multiple threads, and the network throughput is greatly improved.

Description

Technical Field

[0001] This invention relates to the field of video data distribution, and more specifically, to a high-performance video data distribution method. Background Technology

[0002] In video surveillance platforms, the video playback function uses a transcoding service that distributes video data to browsers in a multi-threaded model. If a single thread continuously distributes data packets to a large number of browsers, it creates a single-threaded forwarding performance bottleneck. If each browser has its own distribution thread, there will be numerous thread switches during data distribution, which will severely impact server performance.

[0003] Now consider this scenario: a camera is crucial, and 10,000 users are simultaneously watching its video. When distributing a data packet, would it require a maximum of 10,000 thread switches before sending the data? Such frequent thread switching is unsustainable. Therefore, reducing frequent switching between multiple threads is essential for smooth video playback. Summary of the Invention

[0004] This invention addresses the technical problems existing in the prior art by providing a high-performance video data distribution method, comprising:

[0005] Multiple threads are deployed across multiple CPU cores of the transcoding server;

[0006] Based on video playback requests from multiple user clients, the video playback requests from the multiple user clients are distributed to multiple threads of the transcoding server, so that each thread distributes the video transmitted by the camera to the corresponding client browser according to the allocated video playback request from the user client.

[0007] Based on the above technical solution, the present invention can also be improved as follows.

[0008] Optionally, the step of deploying multiple threads across multiple CPU cores of the transcoding server includes:

[0009] One or more threads are deployed on each CPU core.

[0010] Optionally, the video playback requests from the multiple user clients are distributed to multiple threads of the transcoding server, including:

[0011] The video playback requests from the multiple user clients are evenly distributed across multiple threads of the transcoding server;

[0012] Alternatively, based on the utilization rate of each CPU core of the transcoding server, the video playback requests from the multiple user clients can be evenly distributed across multiple threads of the transcoding server.

[0013] Optionally, the step of evenly distributing the video playback requests from the multiple user clients across multiple threads of the transcoding server based on the utilization rate of each CPU core of the transcoding server includes:

[0014] The utilization rate of each CPU core on the transcoding server is periodically obtained. When a video playback request is received from a user client, the request is preferentially allocated to a thread on a CPU core with low utilization.

[0015] Optional, also includes:

[0016] The utilization rate of each CPU core is periodically retrieved. When the maximum utilization rate of all CPU cores is continuously detected to be 10% greater than the minimum utilization rate, 5% of the video playback requests from user clients on the maximum thread are allocated to the thread with the minimum utilization rate.

[0017] Optionally, the step of distributing the video playback requests from multiple user clients to multiple threads of the transcoding server based on these requests further includes:

[0018] The number of videos played in the history of all user clients is obtained based on sampling, and the median number M of the number of videos played is taken.

[0019] Based on each user's historical video playback habits, predict the total number of video playbacks S for all user clients;

[0020] Based on the median M and the number of video plays S, the predicted number of videos to be played E is calculated.

[0021] Based on the predicted number of videos E to be played, the transcoding server starts threads in advance and preloads the video streams.

[0022] Optionally, obtain the total number of videos played in the history of all user clients based on sampling, and take the median M of the number of videos played, including:

[0023] Based on the number of video playbacks collected from multiple sampling points during the same time period each week, the median M of the number of video playbacks obtained from the multiple sampling points is calculated.

[0024] Optionally, predicting the total number of video plays S for all user clients based on their historical video playback habits includes:

[0025] Record the login method of each user client, and under each login method, the camera ID corresponding to the video played by each user client;

[0026] Based on the camera ID corresponding to the video played by each user client, count the N cameras playing the video for each user client, i.e., the number of videos played.

[0027] Get the number of currently logged-in user clients, and calculate the total number of video plays S for all user clients based on the number of videos played N for each user client.

[0028] Optionally, the step of calculating the predicted number of videos to be played E based on the median M and the number of video plays S includes:

[0029] Calculate the average of the median M and the number of video plays S: E = (M + S) / 2.

[0030] Optional, also includes:

[0031] For each login method, the three cameras with the most clicks for playback by each user client are identified as the user client's hotspot cameras.

[0032] When the user client does not request video, a corresponding thread is pre-allocated to the user client, and the video stream from the hotspot camera is requested.

[0033] If the user client requests the video, the video stream will be sent directly to the user client; if the user client does not request the video for more than the set time, the requested video stream will be closed.

[0034] The present invention provides a high-performance video data distribution method that deploys all threads on multiple CPU cores of a transcoding server. Multiple CPU cores can process data in parallel, which reduces the number of thread switching while using multiple threads to distribute data, thus greatly improving network throughput. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the video stream transmission process;

[0036] Figure 2 A flowchart of a high-performance video data distribution method provided by the present invention;

[0037] Figure 3 A diagram illustrating the distribution of data to the transcoding server. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined with each other to form feasible technical solutions. Such combinations are not constrained by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0039] See Figure 1 This diagram illustrates the entire video platform's video data flow. When a user plays camera video through a browser, the data flow includes: the camera sending RTP (Real-time Transport Protocol) data to the streaming media server; the streaming media server sending RTP data to the transcoding server; the transcoding server processing the video data to generate FMP4 data; and the transcoding server sending the FMP4 data to the browser for playback.

[0040] Figure 2 A flowchart of a high-performance video data distribution method provided by the present invention is shown below. Figure 2 As shown, the method includes:

[0041] Step 1: Deploy multiple threads across multiple CPU cores on the transcoding server.

[0042] As an example, the step of deploying multiple threads on multiple CPU cores of the transcoding server includes: deploying one or more threads on each CPU core.

[0043] Understandably, a transcoding server has multiple CPU cores, and multiple threads are deployed on each CPU core. These threads can be deployed in parallel, either one thread per CPU core or multiple threads per CPU core. Threads on multiple CPU cores can process data in parallel, while threads on the same CPU core can only run in a round-robin fashion.

[0044] Step 2: Based on the video playback requests from multiple user clients, distribute the video playback requests from the multiple user clients to multiple threads of the transcoding server, so that each thread distributes the video transmitted by the camera to the corresponding client browser according to the assigned video playback request from the user client.

[0045] As an example, distributing the video playback requests from the multiple user clients to multiple threads of the transcoding server includes: distributing the video playback requests from the multiple user clients evenly to multiple threads of the transcoding server; or, distributing the video playback requests from the multiple user clients evenly to multiple threads of the transcoding server according to the utilization rate of each CPU core of the transcoding server.

[0046] See also Figure 3 If the transcoding server has 32 CPU cores (or other numbers of cores), and the number of user clients is small, one thread can distribute video data to 50 user clients, and 32 threads can distribute simultaneously, meaning it can support 50*32 users playing video at the same time.

[0047] As the number of user clients increases, more threads are needed, leading to frequent thread switching. This invention avoids increasing the number of threads; instead, it binds 32 threads to 32 CPU cores, reducing thread switching. New user playback requests are evenly distributed across these 32 threads. When the transcoding server is not deploying other services, all 32 CPU cores can run simultaneously, and the parallel data distribution by the 32 threads eliminates thread switching, reducing CPU utilization.

[0048] It should be noted that distributing video playback requests equally among user clients has some drawbacks:

[0049] One drawback is that the videos played by 10,000 users are not from the same source. Different videos have different frame rates and bitrates, so even if users are evenly distributed across 32 cores, some CPU cores will have high utilization while others will have low utilization.

[0050] The second shortcoming is that with 10,000 users playing videos, there will inevitably be some users whose playback stops. After the allocation is completed according to the above steps, some users will stop playing, resulting in some CPU cores having high CPU usage due to the number of users playing, while others have low CPU usage due to fewer users playing.

[0051] The third shortcoming is that if the server is running other programs at the same time, the other programs will also consume the usage of a certain CPU core, which may result in a situation where the usage of one CPU core is high and the usage of another CPU core is low.

[0052] When a CPU core reaches 100% utilization, the threads on that core will get stuck, causing all users to experience video playback stuttering and interruption.

[0053] To address the above shortcomings, this invention proposes an improved solution: dynamically allocating threads based on the utilization rate of each CPU core of the transcoding server for video playback requests from user clients.

[0054] As an example, the step of evenly distributing the video playback requests from multiple user clients to multiple threads of the transcoding server based on the utilization rate of each CPU core of the transcoding server includes: periodically obtaining the utilization rate of each CPU core on the transcoding server, and when a video playback request from a user client is received, prioritizing the allocation of the user client's video playback request to a thread in a CPU core with a low utilization rate.

[0055] Specifically, the utilization rate of each CPU core is periodically obtained. When the maximum utilization rate of all CPU cores is continuously detected to be 10% greater than the minimum utilization rate, 5% of the video playback requests from user clients on the maximum thread are allocated to the thread with the minimum utilization rate.

[0056] Understandably, a function to obtain CPU core utilization is added to the bound thread to periodically obtain the utilization of each CPU core in the transcoding server. When multiple user clients request to watch videos, instead of distributing users evenly across these threads, users are preferentially assigned to the thread with lower CPU core utilization.

[0057] The CPU core utilization is checked every 2 seconds. If the maximum utilization of all CPU cores is more than 10% greater than the minimum utilization after more than 3 checks, then 5% of the video playback requests from user clients on the thread with the highest utilization are allocated to the thread with the lowest utilization. This achieves dynamic allocation of video playback requests from user clients, thereby making the utilization of each CPU core relatively stable.

[0058] As an example, the step of allocating video playback requests from multiple user clients to multiple threads of the transcoding server based on these requests includes, prior to: obtaining the historical video playback counts of all user clients based on sampling, and taking the median M of these counts; predicting the total number of video playbacks S for all user clients based on their historical video playback habits; calculating the predicted number of videos to be played E based on the median M and the video playback count S; and preloading the video streams in advance based on the predicted number of videos to be played E.

[0059] Understandably, the system learns to dynamically allocate threads and preload video streams based on three factors: the user's client login playback time, the user's client using a computer browser / mobile client (cellular data connection / Wi-Fi connection), and the hotspot camera.

[0060] The process involves obtaining the historical video playback count of all user clients based on sampling, and then taking the median M of the video playback count. This includes: collecting the video playback count of all user clients within the same time period each week based on multiple sampling points, and calculating the median M of the video playback count obtained from multiple sampling points.

[0061] Understandably, the login and video playback times for each user client are recorded and stored in the database. For example, if the reporting time is 9 AM, there will be a surge in user client logins and video playback concurrency, resulting in a high volume of activity. Based on the previous month's access data, the number of user clients logging in and playing videos at 9 AM that morning is predicted. The median number of videos played at 9 AM on a given Monday is calculated using the data from previous weeks (e.g., Mondays) as the parameter M. A sampling point is established every half hour, resulting in 7 * 24 * 2 sampling points per week. The number of videos played at each sampling point is obtained as the sample value. Five minutes before the sampling point, the number of videos played at that sampling point from the previous 10 weeks is used to calculate the median number of plays, M.

[0062] As an example, predicting the number of video plays S for all user clients based on their historical video playback habits includes: recording the login method of each user client and the camera ID corresponding to the video played by each user client under each login method; counting the N cameras played by each user client, i.e., the number of videos played, based on the camera ID corresponding to the video played by each user client; obtaining the number of currently logged-in user clients and calculating the number of video plays S for all user clients based on the number of videos played N by each user client.

[0063] Understandably, when a user clicks to play a video, the system tracks the user's login method. This includes login via computer browser or mobile network (cellular / Wi-Fi). Users are more likely to play videos when logging in via computer browser or mobile network (Wi-Fi), while the probability of mobile network login via cellular network is lower. The system also records each user's operating habits and the camera ID corresponding to the video played by each user's client under each login mode. Each user's client will simultaneously play videos from N cameras, meaning the number of video playbacks is N.

[0064] Count the number of currently logged-in user clients. The number of videos playing simultaneously is the sum of the video playback counts of all user clients, S = (N1 + N2 + N3 + ... + Ncount).

[0065] Based on the median M and the number of video plays S, the predicted number of videos to be played E is calculated, including: calculating the average of the median M and the number of video plays S, E = (M + S) / 2.

[0066] Based on the predicted number of video streams E to be played, the transcoding server knows in advance how many video streams to play and starts threads accordingly, with each thread having a default capacity of 50 video streams. This prediction is made every half hour, and the predicted values ​​are then sent to the transcoding service, which starts its threads in advance.

[0067] As an example, the method further includes: identifying the three cameras with the most clicks for playback by each user client under each login method, and designating them as the user client's hotspot cameras; pre-allocating a corresponding thread for the user client when the user client is not requesting playback, and requesting the video stream from the hotspot cameras; sending the video stream directly to the user client when the user client requests playback; and closing the requested video stream if the user client does not request playback for a set time.

[0068] Understandably, when recording user client video playback habits, it is necessary to record which video each user client plays each time, then count the number of times each user client plays each camera video, create a hotspot camera table for each user client, and record the user client login method (user login methods include computer client browser login, mobile client login via cellular network / Wi-Fi), and record the user client's operation habits when clicking to play videos.

[0069] When a user client is logged in, the login method is first determined. The three cameras with the most views under this login method are the user client's "hotspot cameras." If the user client is not viewing the video stream, a thread is allocated and the video stream is requested, but not sent to the user client. Once the user client requests the video stream, it is sent directly to the user client, speeding up the video loading process (normal request loads in 3 seconds, pre-loading loads in 1 second) and improving the user experience. If the user client does not view the video stream for 10 minutes, the video stream is closed.

[0070] This invention provides a high-performance video data distribution method, which has the following advantages:

[0071] (1) For security video platform browsers playing videos, the transcoding server distributes video stream data to the browser. When the number of users exceeds the set value (number of CPU cores * 50), the service performance is improved by binding threads of the same number of cores to the CPU cores to reduce frequent thread switching.

[0072] (2) Dynamic allocation solves the problem of high utilization of a certain CPU core and low utilization of other CPU cores. It ensures that the difference between the maximum and minimum utilization of all CPU cores does not exceed 10%, which greatly improves the stability of program operation.

[0073] (3) Based on this thread allocation, the system also learns to dynamically allocate threads and preload video streams based on three factors: user client login playback time, user's computer browser / mobile client (cellular data connection / wifi connection), and hotspot camera.

[0074] (4) All threads are deployed on multiple CPU cores of the transcoding server. Multiple CPU cores can process in parallel, which will reduce the number of thread switching and at the same time, multiple threads can be used to distribute data, greatly improving network throughput.

[0075] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0076] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0077] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0078] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0079] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0080] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0081] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A high-performance video data distribution method, characterized in that, include: Multiple threads are deployed across multiple CPU cores of the transcoding server; Based on video playback requests from multiple user clients, the video playback requests from the multiple user clients are distributed to multiple threads of the transcoding server, so that each thread distributes the video transmitted by the camera to the corresponding client browser according to the distributed video playback requests from the user clients. The step of allocating video playback requests from multiple user clients to multiple threads of the transcoding server based on these requests also includes: The number of videos played in the history of all user clients is obtained based on sampling, and the median number M of the number of videos played is taken. Based on each user's historical video playback habits, predict the total number of video playbacks S for all user clients; Based on the median M and the number of video plays S, the predicted number of videos to be played E is calculated. Based on the predicted number of videos E to be played, the transcoding server starts threads in advance and preloads the video streams.

2. The high-performance video data distribution method according to claim 1, characterized in that, The method of deploying multiple threads across multiple CPU cores of the transcoding server includes: One or more threads are deployed on each CPU core.

3. The high-performance video data distribution method according to claim 1, characterized in that, Distributing video playback requests from the multiple user clients to multiple threads of the transcoding server includes: The video playback requests from the multiple user clients are evenly distributed across multiple threads of the transcoding server; Alternatively, the utilization rate of each CPU core on the transcoding server can be obtained periodically, and when a video playback request is received from a user client, the video playback request from the user client can be preferentially allocated to a thread in a CPU core with low utilization.

4. The high-performance video data distribution method according to claim 3, characterized in that, Also includes: The utilization rate of each CPU core is periodically retrieved. When the maximum utilization rate of all CPU cores is continuously detected to be 10% greater than the minimum utilization rate, 5% of the video playback requests from user clients on the maximum thread are allocated to the thread with the minimum utilization rate.

5. The high-performance video data distribution method according to claim 1, characterized in that, The number of videos played in the history of all user clients is obtained based on sampling, and the median M of the number of videos played is taken, including: Based on the number of video playbacks collected from multiple sampling points during the same time period each week, the median M of the number of video playbacks obtained from the multiple sampling points is calculated.

6. The high-performance video data distribution method according to claim 1, characterized in that, The step of predicting the total number of video plays S for all user clients based on each user client's historical video playback habits includes: Record the login method of each user client, and under each login method, the camera ID corresponding to the video played by each user client; Based on the camera ID corresponding to the video played by each user client, count the N cameras playing the video for each user client, i.e., the number of videos played, where N is a positive integer; Get the number of currently logged-in user clients, and calculate the total number of video plays S for all user clients based on the number of videos played N for each user client.

7. The high-performance video data distribution method according to claim 1, characterized in that, The calculation of the predicted number of videos to be played, based on the median M and the number of video plays S, includes: Calculate the average of the median M and the number of video plays S: E = (M + S) / 2.

8. The high-performance video data distribution method according to claim 6, characterized in that, Also includes: For each login method, the three cameras with the most clicks for playback by each user client are identified as the user client's hotspot cameras. When the user client does not request video, a corresponding thread is pre-allocated to the user client, and the video stream from the hotspot camera is requested. If the user client requests the video, the video stream will be sent directly to the user client; if the user client does not request the video for more than the set time, the requested video stream will be closed.

Images