Video stream processing method and apparatus, electronic device, and storage medium

Abstract

The application provides a video stream processing method and device, electronic equipment and storage medium. The method comprises the following steps: based on a preset window, scheduling a plurality of first video streams in a time slice rotation manner to determine a plurality of video stream slices corresponding to a target time slice; inputting the plurality of video stream slices corresponding to the target time slice into an algorithm engine, and the algorithm engine is used for analyzing and reasoning the video stream slices to obtain a reasoning result. The plurality of first video streams are scheduled in the time slice rotation manner, the plurality of video stream slices corresponding to the target time slice are determined, and then the algorithm engine is used for analyzing and reasoning the video stream slices to obtain the reasoning result. Since the size of the preset window is determined based on the computing power performance of the server, the total number of the plurality of video stream slices is less than or equal to the size of the preset window, so that the algorithm engine can process the plurality of video stream slices in the target time slice, and the stable operation of the edge server can be ensured.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a video stream processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] For a single edge server, the computing power and decoding capabilities required in traditional edge computing scenarios do not exceed the limit of a single server. However, due to cost considerations, purchasers and contractors are increasingly inclined to adopt solutions that overload a single edge server to process video streams. That is, too many video stream devices (usually exceeding the computing power of a single edge server) are often connected to a single server for inference and analysis.

[0003] Simultaneously connecting all video streaming devices for reading and analysis often triggers serious system errors, such as insufficient decoding capabilities causing the entire system to block. Ensuring the stable operation of edge servers when a large number of video streaming devices are connected is a pressing issue that the industry needs to address. Summary of the Invention

[0004] To address the problems existing in the prior art, embodiments of the present invention provide a video stream processing method, apparatus, electronic device, and storage medium.

[0005] In a first aspect, the present invention provides a video stream processing method, comprising:

[0006] Based on a preset window, multiple first video streams are scheduled using a time-slice rotation method to determine multiple video stream slices corresponding to a target time slice. The total number of the multiple video stream slices is less than or equal to the size of the preset window, which is determined based on the server's computing power performance.

[0007] Multiple video stream slices corresponding to the target time slice are input into the algorithm engine, which is used to analyze and reason about the video stream slices to obtain reasoning results.

[0008] Optionally, according to a video stream processing method provided by the present invention, the step of scheduling multiple first video streams based on a preset window and using a time-slice rotation method to determine multiple video stream slices corresponding to a target time slice includes:

[0009] When the time slice rotation mode is the first rotation mode, the multiple first video streams are scheduled based on a preset window using the first rotation mode to determine the multiple video stream slices corresponding to the target time slice;

[0010] Alternatively, if the time slice rotation method is the second rotation method, the multiple first video streams are scheduled based on a preset window using the second rotation method to determine the multiple video stream slices corresponding to the target time slice;

[0011] In the first round-robin mode, the scheduling probability of each video stream is equal; in the second round-robin mode, the scheduling probability of a higher-priority video stream is higher than that of a lower-priority video stream.

[0012] Optionally, according to a video stream processing method provided by the present invention, the step of scheduling the multiple first video streams based on a preset window and using a first round-robin method to determine the multiple video stream slices corresponding to the target time slice includes:

[0013] Based on the sorting of the multiple first video streams, the first position of the preset window is determined;

[0014] Based on the sliding step size, the preset window is slid according to the sorting of the multiple first video streams to determine the second position of the preset window;

[0015] Based on the second position of the preset window and the size of the preset window, multiple video stream slices corresponding to the target time slice are determined.

[0016] Optionally, according to a video stream processing method provided by the present invention, the step of scheduling the multiple first video streams based on a preset window and using a second round-robin method to determine the multiple video stream slices corresponding to the target time slice includes:

[0017] Based on the priority of each first video stream, determine the second video stream with the highest priority and the slices of each second video stream;

[0018] The remaining amount of the preset window is determined based on the size of the preset window and the number of the second video stream with the highest priority.

[0019] Based on the remaining amount of the preset window, at least one third video stream is scheduled through the first round-robin method to determine at least one scheduled fourth video stream and the slices of each fourth video stream. The at least one third video stream is a video stream other than the second video stream with the highest priority among the multiple first video streams.

[0020] Based on the slices of each second video stream and each fourth video stream, multiple video stream slices corresponding to the target time slice are determined.

[0021] Optionally, according to a video stream processing method provided by the present invention, before scheduling multiple first video streams based on a preset window and determining multiple video stream slices corresponding to a target time slice by time-slice rotation, the method further includes:

[0022] Determine whether the priorities corresponding to the multiple first video streams are the same;

[0023] If it is determined that the multiple first video streams correspond to the same priority, then the time slice rotation method is determined to be the first rotation method;

[0024] Alternatively, if it is determined that the priorities of the multiple first video streams are not the same, then the time slice rotation method is determined to be the second rotation method.

[0025] Optionally, according to a video stream processing method provided by the present invention, before scheduling multiple first video streams based on a preset window and determining multiple video stream slices corresponding to a target time slice by time-slice rotation, the method further includes:

[0026] Based on the server's computing power and time slice length, determine the maximum number of video stream slices that the server can process within the time slice length.

[0027] Based on the maximum number of slices and a preset ratio value, the size of the preset window is determined, whereby the preset ratio value represents the proportional relationship between the size of the preset window and the maximum number of slices.

[0028] Optionally, according to a video stream processing method provided by the present invention, before scheduling multiple first video streams based on a preset window and determining multiple video stream slices corresponding to a target time slice by time-slice rotation, the method further includes:

[0029] Obtain the operating status of each video stream device, wherein the operating status is either power-on or power-off;

[0030] Based on the operating status of each video streaming device, multiple target video streaming devices that are powered on are identified;

[0031] Based on the video streams corresponding to the multiple target video stream devices, the multiple first video streams are determined.

[0032] In a second aspect, the present invention also provides a video stream processing apparatus, comprising:

[0033] The determination module is used to schedule multiple first video streams based on a preset window and in a time-slice rotation manner, and determine multiple video stream slices corresponding to a target time slice. The total number of the multiple video stream slices is less than or equal to the size of the preset window, and the size of the preset window is determined based on the computing power performance of the server.

[0034] The interface module is used to input multiple video stream slices corresponding to the target time slice to the algorithm engine, which is used to analyze and reason about the video stream slices to obtain reasoning results.

[0035] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the video stream processing method described above.

[0036] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the video stream processing method as described above.

[0037] The video stream processing method, apparatus, electronic device, and storage medium provided by this invention schedule multiple first video streams through a time-slice rotation method. This allows for the determination of multiple video stream slices corresponding to a target time slice. These multiple video stream slices can then be input into an algorithm engine, which analyzes and infers the video stream slices to obtain inference results. Since the size of the preset window is determined based on the server's computing power, the total number of multiple video stream slices is less than or equal to the size of the preset window. This ensures that the algorithm engine processes all the video stream slices within the target time slice, preventing insufficient decoding capabilities from causing system congestion. This also ensures the stable operation of the edge server even when many video stream devices are connected to it. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0039] Figure 1 This is one of the flowcharts illustrating the video stream processing method provided by the present invention;

[0040] Figure 2 This is the second flowchart of the video stream processing method provided by the present invention;

[0041] Figure 3This is the third flowchart of the video stream processing method provided by the present invention;

[0042] Figure 4 This is the fourth flowchart of the video stream processing method provided by the present invention;

[0043] Figure 5 This is a schematic diagram of the time-division non-priority video stream scheduling and distribution logic provided by the present invention;

[0044] Figure 6 This is the fifth flowchart of the video stream processing method provided by the present invention;

[0045] Figure 7 This is a schematic diagram of the video stream scheduling and distribution logic based on time-division priority provided by the present invention;

[0046] Figure 8 This is the sixth flowchart of the video stream processing method provided by the present invention;

[0047] Figure 9 This is the seventh flowchart of the video stream processing method provided by the present invention;

[0048] Figure 10 This is the eighth flowchart of the video stream processing method provided by the present invention;

[0049] Figure 11 This is the ninth flowchart of the video stream processing method provided by the present invention;

[0050] Figure 12 This is one of the structural schematic diagrams of the video stream processing device provided by the present invention;

[0051] Figure 13 This is the second schematic diagram of the video stream processing device provided by the present invention;

[0052] Figure 14 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0054] Figure 1 This is one of the flowcharts illustrating the video stream processing method provided by the present invention, such as... Figure 1As shown, the execution entity of the video stream processing method can be an electronic device, such as an edge server. The method includes:

[0055] Step 101: Based on a preset window, schedule multiple first video streams using a time-slice rotation method to determine multiple video stream slices corresponding to the target time slice. The total number of the multiple video stream slices is less than or equal to the size of the preset window, which is determined based on the computing power performance of the server.

[0056] Specifically, in order to ensure the stable operation of the edge server when too many video stream devices are connected to the edge server, multiple first video streams can be scheduled by time-slice round-robin. Multiple video stream slices corresponding to the target time slice can be determined, and then the multiple video stream slices corresponding to the target time slice can be input into the algorithm engine.

[0057] Step 102: Input multiple video stream slices corresponding to the target time slice into the algorithm engine. The algorithm engine is used to analyze and reason about the video stream slices to obtain reasoning results.

[0058] Specifically, after inputting multiple video stream slices corresponding to the target time slice into the algorithm engine, the algorithm engine can analyze and reason about the video stream slices to obtain the reasoning results.

[0059] Understandably, since the size of the preset window is determined based on the server's computing power, the total number of video stream slices is less than or equal to the size of the preset window. This ensures that the algorithm engine processes all the video stream slices within the target time slice, preventing insufficient decoding capabilities from causing system congestion. Scheduling multiple video streams through time-slice rotation enables a single edge server to perform video stream inference under overload conditions, ensuring stable operation of the edge server while improving the computing power utilization efficiency of a single server.

[0060] Optionally, according to a video stream processing method provided by the present invention, the step of scheduling multiple first video streams based on a preset window and using a time-slice rotation method to determine multiple video stream slices corresponding to a target time slice includes:

[0061] When the time slice rotation mode is the first rotation mode, the multiple first video streams are scheduled based on a preset window using the first rotation mode to determine the multiple video stream slices corresponding to the target time slice;

[0062] Alternatively, if the time slice rotation method is the second rotation method, the multiple first video streams are scheduled based on a preset window using the second rotation method to determine the multiple video stream slices corresponding to the target time slice;

[0063] In the first round-robin mode, the scheduling probability of each video stream is equal; in the second round-robin mode, the scheduling probability of a higher-priority video stream is higher than that of a lower-priority video stream.

[0064] Specifically, Figure 2 This is a second flowchart illustrating the video stream processing method provided by the present invention, as shown below. Figure 2 As shown, the video stream processing method includes steps 201 to 202.

[0065] Step 201: When the time slice rotation mode is the first rotation mode, based on the preset window, the first rotation mode is used to schedule multiple first video streams to determine the multiple video stream slices corresponding to the target time slice. In the first rotation mode, the scheduling probability of each video stream is equal.

[0066] Understandably, the first round-robin method can be a time-sharing, non-priority round-robin scheduling method where each video stream has the same priority (each video stream has the same scheduling probability) and video streams can be distributed by sliding a preset window.

[0067] Step 202: Input multiple video stream slices corresponding to the target time slice into the algorithm engine.

[0068] Therefore, by using a non-priority round-robin scheduling method to schedule and distribute video streams, a reasonable number of video streams can be connected to the algorithm inference engine for real-time analysis and inference. This enables a single edge server to perform video stream inference under overload conditions, thereby improving the efficiency of computing power utilization of a single server.

[0069] Specifically, Figure 3 This is the third flowchart of the video stream processing method provided by the present invention, as shown below. Figure 3 As shown, the video stream processing method includes steps 301 to 302.

[0070] Step 301: When the time slice rotation mode is the second rotation mode, based on the preset window, the multiple first video streams are scheduled through the second rotation mode to determine the multiple video stream slices corresponding to the target time slice. In the second rotation mode, the scheduling probability of the video stream with higher priority is higher than that of the video stream with lower priority.

[0071] Understandably, the second round-robin method can be a time-sharing priority round-robin scheduling method, where each video stream is configured with a priority, and the time slice includes the highest priority video stream and one or more low priority video streams. The low priority video streams can be selected from all low priority video streams through round-robin analysis.

[0072] Step 302: Input multiple video stream slices corresponding to the target time slice into the algorithm engine. The algorithm engine is used to analyze and reason about the video stream slices to obtain reasoning results.

[0073] Therefore, by using a priority-based round-robin scheduling method to schedule and distribute video streams, a reasonable number of video streams can be connected to the algorithm inference engine for real-time analysis and inference. This enables a single edge server to perform video stream inference under overload conditions, thereby improving the efficiency of computing power utilization of a single server.

[0074] Optionally, according to a video stream processing method provided by the present invention, the step of scheduling the multiple first video streams based on a preset window and using a first round-robin method to determine the multiple video stream slices corresponding to the target time slice includes:

[0075] Based on the sorting of the multiple first video streams, the first position of the preset window is determined;

[0076] Based on the sliding step size, the preset window is slid according to the sorting of the multiple first video streams to determine the second position of the preset window;

[0077] Based on the second position of the preset window and the size of the preset window, multiple video stream slices corresponding to the target time slice are determined.

[0078] Specifically, Figure 4 This is the fourth flowchart of the video stream processing method provided by the present invention, as shown below. Figure 4 As shown, the video stream processing method includes steps 401 to 404.

[0079] Step 401: Determine the first position of the preset window based on the sorting of the multiple first video streams.

[0080] Optionally, if the target time slice is the first time slice (in the case of the first rotation method), the preset window can be determined to be aligned with the first video stream in the multiple first video streams, that is, the first position.

[0081] Optionally, if the target time slice is not the first time slice, processing of the video stream slices corresponding to the target time slice can begin after processing of the multiple video stream slices corresponding to the previous time slice. The current position of the preset window, i.e., the first position, can be determined by analyzing the order of the multiple video stream slices corresponding to the previous time slice and the multiple first video streams. The previous time slice is the time slice adjacent to the target time slice and preceding it in time sequence.

[0082] Step 402: Based on the sliding step size, slide the preset window according to the sorting of the multiple first video streams to determine the second position of the preset window.

[0083] It is understandable that after determining the first position of the preset window, the first position can be used as the starting point for sliding, and then the preset window can be slid according to the sorting of the multiple first video streams based on the sliding step size. The endpoint after sliding is the second position of the preset window.

[0084] Step 403: Based on the second position of the preset window and the size of the preset window, determine multiple video stream slices corresponding to the target time slice.

[0085] It is understandable that after sliding the preset window, the video stream selected by the preset window in the multiple first video streams can be analyzed based on the second position and size of the preset window to determine the multiple video stream slices corresponding to the target time slice.

[0086] Step 404: Input multiple video stream slices corresponding to the target time slice into the algorithm engine.

[0087] Optionally, Figure 5 This is a schematic diagram of the time-division non-priority video stream scheduling and distribution logic provided by the present invention, as shown below. Figure 5 As shown, the first round-robin method can be a time-sharing, non-priority round-robin scheduling method where all video streams have the same priority and can be distributed using a sliding window algorithm.

[0088] Therefore, by using a non-priority round-robin scheduling method to schedule and distribute video streams, a reasonable number of video streams can be connected to the algorithm inference engine for real-time analysis and inference. This enables a single edge server to perform video stream inference under overload conditions, thereby improving the efficiency of computing power utilization of a single server.

[0089] Optionally, according to a video stream processing method provided by the present invention, the step of scheduling the multiple first video streams based on a preset window and using a second round-robin method to determine the multiple video stream slices corresponding to the target time slice includes:

[0090] Based on the priority of each first video stream, determine the second video stream with the highest priority and the slices of each second video stream;

[0091] The remaining amount of the preset window is determined based on the size of the preset window and the number of the second video stream with the highest priority.

[0092] Based on the remaining amount of the preset window, at least one third video stream is scheduled through the first round-robin method to determine at least one scheduled fourth video stream and the slices of each fourth video stream. The at least one third video stream is a video stream other than the second video stream with the highest priority among the multiple first video streams.

[0093] Based on the slices of each second video stream and each fourth video stream, multiple video stream slices corresponding to the target time slice are determined.

[0094] Specifically, Figure 6 This is the fifth flowchart of the video stream processing method provided by the present invention, as shown below. Figure 6 As shown, the video stream processing method includes steps 601 to 605.

[0095] Step 601: Based on the priority of each first video stream, determine the second video stream with the highest priority and the slices of each second video stream.

[0096] Understandably, it is possible to compare the priorities of each first video stream to determine the second video stream with the highest priority (which can be one or more second video streams), and also to determine the slices of each second video stream.

[0097] Step 602: Determine the remaining amount of the preset window based on the size of the preset window and the number of the second video stream with the highest priority.

[0098] Understandably, after determining the second video stream with the highest priority, the total number of the second video stream with the highest priority can be calculated, and then the size of the preset window can be subtracted from the total number to determine the remaining amount of the preset window.

[0099] Step 603: Based on the remaining amount of the preset window, at least one third video stream is scheduled through the first round-robin method to determine at least one scheduled fourth video stream and the slices of each fourth video stream.

[0100] It is understandable that after determining the remaining amount of the preset window, a temporary sliding window can be determined based on the remaining amount. The size of the temporary sliding window is equal to the remaining amount. Then, based on the temporary sliding window, at least one third video stream (video streams other than the highest priority second video stream among multiple first video streams) can be scheduled through the first round-robin method to determine at least one scheduled fourth video stream and the slices of each fourth video stream.

[0101] Step 604: Based on the slices of each second video stream and each fourth video stream, determine multiple video stream slices corresponding to the target time slice.

[0102] Step 605: Input multiple video stream slices corresponding to the target time slice into the algorithm engine.

[0103] Optionally, Figure 7 This is a schematic diagram of the video stream scheduling and distribution logic based on time-division priority provided by the present invention, as shown below. Figure 7As shown, the second round-robin scheduling method can be a time-sharing priority round-robin scheduling method, where each video stream is configured with a priority, and the time slice includes the highest priority video stream (e.g., ...). Figure 7 The video stream with a medium priority of Pri0 and one or more low priority video streams (such as...) Figure 7 The video stream with a medium priority of Pri1 is selected from all low-priority video streams through a round-robin analysis. (Pri0 has a higher priority than Pri1.)

[0104] Therefore, by using a priority-based round-robin scheduling method to schedule and distribute video streams, a reasonable number of video streams can be connected to the algorithm inference engine for real-time analysis and inference. This enables a single edge server to perform video stream inference under overload conditions, thereby improving the efficiency of computing power utilization of a single server.

[0105] Optionally, according to a video stream processing method provided by the present invention, before scheduling multiple first video streams based on a preset window and determining multiple video stream slices corresponding to a target time slice by time-slice rotation, the method further includes:

[0106] Determine whether the priorities corresponding to the multiple first video streams are the same;

[0107] If it is determined that the multiple first video streams correspond to the same priority, then the time slice rotation method is determined to be the first rotation method;

[0108] Alternatively, if it is determined that the priorities of the multiple first video streams are not the same, then the time slice rotation method is determined to be the second rotation method.

[0109] Specifically, Figure 8 This is the sixth flowchart of the video stream processing method provided by the present invention, as shown below. Figure 8 As shown, the video stream processing method includes steps 801 to 805.

[0110] Step 801: Determine whether the priorities of the multiple first video streams are the same. If they are, proceed to step 802; otherwise, proceed to step 803.

[0111] Step 802: Determine the time slice rotation mode as the first rotation mode.

[0112] Step 803: Determine the time slice rotation mode as the second rotation mode.

[0113] Step 804: Based on a preset window, schedule multiple first video streams using a time-slice rotation method to determine multiple video stream slices corresponding to the target time slice.

[0114] Step 805: Input multiple video stream slices corresponding to the target time slice into the algorithm engine.

[0115] Understandably, during server operation, multiple video streams with equal priorities may be adjusted to multiple video streams with unequal priorities, or vice versa. To adapt to such adjustment scenarios, before determining the multiple video stream slices corresponding to the target time slice, the priority of each of the multiple first video streams can be checked to determine whether the time slice rotation method is the first rotation method or the second rotation method. This can support scenarios where video stream priorities can be flexibly adjusted, improving the applicability of the solution.

[0116] Optionally, according to a video stream processing method provided by the present invention, before scheduling multiple first video streams based on a preset window and determining multiple video stream slices corresponding to a target time slice by time-slice rotation, the method further includes:

[0117] Based on the server's computing power and time slice length, determine the maximum number of video stream slices that the server can process within the time slice length.

[0118] Based on the maximum number of slices and a preset ratio value, the size of the preset window is determined, whereby the preset ratio value represents the proportional relationship between the size of the preset window and the maximum number of slices.

[0119] Specifically, Figure 9 This is the seventh flowchart of the video stream processing method provided by the present invention, as shown below. Figure 9 As shown, the video stream processing method includes steps 901 to 904.

[0120] Step 901: Based on the server's computing power and time slice length, determine the maximum number of video stream slices that the server can process within the time slice length.

[0121] Understandably, in order to ensure that the server can process multiple video stream slices within the target time slice, the maximum number of video stream slices that the server can process within the time slice length can be calculated based on the server's computing power and the time slice length.

[0122] Step 902: Determine the size of the preset window based on the maximum number of slices and the preset ratio value.

[0123] Understandably, a preset ratio value can be set, which can represent the ratio between the size of the preset window and the maximum number of slices. Then, the product of the maximum number of slices and the preset ratio value can be calculated, and the product can be used as the size of the preset window.

[0124] Optionally, the range of the preset ratio value a can be 0 < a < 1, and the product of the maximum number of slices and the preset ratio value a can be determined as the size of the preset window.

[0125] Step 903: Based on a preset window, schedule multiple first video streams using a time-slice rotation method to determine multiple video stream slices corresponding to the target time slice.

[0126] Step 904: Input multiple video stream slices corresponding to the target time slice into the algorithm engine.

[0127] Therefore, by determining the maximum number of video stream slices that the server can process within the time slice length, a suitable size can be set for the preset window. If the total number of multiple video stream slices is less than or equal to the size of the preset window, it can be ensured that the algorithm engine processes multiple video stream slices within the target time slice, avoiding insufficient decoding capability that could cause the entire system to be blocked.

[0128] Optionally, according to a video stream processing method provided by the present invention, before scheduling multiple first video streams based on a preset window and determining multiple video stream slices corresponding to a target time slice by time-slice rotation, the method further includes:

[0129] Obtain the operating status of each video stream device, wherein the operating status is either power-on or power-off;

[0130] Based on the operating status of each video streaming device, multiple target video streaming devices that are powered on are identified;

[0131] Based on the video streams corresponding to the multiple target video stream devices, the multiple first video streams are determined.

[0132] Specifically, Figure 10 This is the eighth flowchart of the video stream processing method provided by the present invention, as shown below. Figure 10 As shown, the video stream processing method includes steps 1001 to 1005.

[0133] Step 1001: Obtain the operating status of each video stream device.

[0134] Understandably, in order to prevent the sudden loss of cameras from causing all algorithms to stop reasoning, the operating status of video streaming devices can be monitored to obtain the operating status of each video streaming device.

[0135] Step 1002: Based on the operating status of each video stream device, determine the multiple target video stream devices that are powered on.

[0136] Understandably, the operating status can indicate whether the video streaming device is in a powered-on or powered-off state. After obtaining the operating status of each video streaming device, multiple video streaming devices can be filtered to obtain multiple target video streaming devices in a powered-on state, so as to avoid the impact of video streaming devices in a powered-off state on the algorithm engine.

[0137] Step 1003: Determine multiple first video streams based on the video streams corresponding to multiple target video stream devices.

[0138] Understandably, the video streams corresponding to multiple target video streaming devices can be used as multiple first video streams.

[0139] Step 1004: Based on a preset window, schedule multiple first video streams using a time-slice rotation method to determine multiple video stream slices corresponding to the target time slice.

[0140] Step 1005: Input multiple video stream slices corresponding to the target time slice into the algorithm engine.

[0141] Therefore, by decoupling the video stream acquisition process from the analysis and reasoning process, it is possible to avoid the sudden loss of one or more cameras, which would cause all algorithms to stop reasoning and improve the stability of server operation.

[0142] Optionally, Figure 11 This is the ninth flowchart of the video stream processing method provided by the present invention, as shown below. Figure 11 As shown, the video stream processing method includes steps 1101 to 1104.

[0143] Step 1101: Load the algorithm inference engine into memory.

[0144] Step 1102: Obtain multiple video streams and schedule the multiple first video streams using a time-slice rotation method based on a preset window to determine the multiple video stream slices corresponding to the target time slice.

[0145] Optionally, multiple first video streams can be scheduled based on a preset window using a first round-robin method to determine multiple video stream slices corresponding to the target time slice.

[0146] Optionally, multiple first video streams can be scheduled based on a preset window using a second round-robin method to determine multiple video stream slices corresponding to the target time slice.

[0147] Step 1103: For multiple video stream slices corresponding to the target time slice, configure the algorithm and video stream based on the algorithm inference engine, and initialize the video stream.

[0148] Understandably, each video stream has a different processing algorithm, so after each rotation, it is necessary to configure the algorithm and video stream, and initialize the video stream.

[0149] Step 1104: Based on the algorithm inference engine and multiple video stream slices corresponding to the target time slice, perform inference on the video stream, obtain the inference result corresponding to the target time slice, and execute step 1102 to perform the next time slice rotation inference.

[0150] Optionally, inference results can be shared and displayed through open interfaces.

[0151] Optionally, determine whether the multiple video stream slices corresponding to the target time slice of this distribution have been analyzed and reasoned. If so, execute step 1102 to perform the next time slice rotation reasoning; otherwise, continue to analyze and reason for the current target time slice.

[0152] The video stream processing method provided by this invention schedules multiple first video streams using a time-slice round-robin approach. This allows for the determination of multiple video stream slices corresponding to a target time slice. These multiple video stream slices are then input into an algorithm engine, which analyzes and infers the video stream slices to obtain inference results. Since the size of the preset window is determined based on the server's computing power, the total number of video stream slices is less than or equal to the size of the preset window. This ensures that the algorithm engine processes all the video stream slices within the target time slice, preventing system congestion due to insufficient decoding capabilities. This also ensures stable operation of the edge server even when many video stream devices are connected to it.

[0153] The video stream processing apparatus provided by the present invention is described below. The video stream processing apparatus described below can be referred to in correspondence with the video stream processing method described above.

[0154] Figure 12 This is one of the structural schematic diagrams of the video stream processing device provided by the present invention, such as... Figure 12 As shown, the video stream processing device includes: a determination module 1201 and an interface module 1202, wherein:

[0155] The determination module 1201 is used to schedule multiple first video streams based on a preset window and in a time-slice rotation manner, and determine multiple video stream slices corresponding to a target time slice. The total number of the multiple video stream slices is less than or equal to the size of the preset window, and the size of the preset window is determined based on the computing power performance of the server.

[0156] The interface module 1202 is used to input multiple video stream slices corresponding to the target time slice to the algorithm engine, and the algorithm engine is used to analyze and reason about the video stream slices to obtain reasoning results.

[0157] The video stream processing device provided by this invention schedules multiple first video streams using a time-slice rotation method. This allows for the determination of multiple video stream slices corresponding to a target time slice. These video stream slices are then input into an algorithm engine, which analyzes and infers the video stream slices to obtain inference results. Since the size of the preset window is determined based on the server's computing power, the total number of video stream slices is less than or equal to the size of the preset window. This ensures that the algorithm engine processes all the video stream slices within the target time slice, preventing system congestion due to insufficient decoding capabilities. This also ensures stable operation of the edge server even when many video stream devices are connected to it.

[0158] Optionally, the determining module is specifically used for:

[0159] When the time slice rotation mode is the first rotation mode, the multiple first video streams are scheduled based on a preset window using the first rotation mode to determine the multiple video stream slices corresponding to the target time slice;

[0160] Alternatively, if the time slice rotation method is the second rotation method, the multiple first video streams are scheduled based on a preset window using the second rotation method to determine the multiple video stream slices corresponding to the target time slice;

[0161] In the first round-robin mode, the scheduling probability of each video stream is equal; in the second round-robin mode, the scheduling probability of a higher-priority video stream is higher than that of a lower-priority video stream.

[0162] Optionally, the determining module is specifically used for:

[0163] Based on the sorting of the multiple first video streams, the first position of the preset window is determined;

[0164] Based on the sliding step size, the preset window is slid according to the sorting of the multiple first video streams to determine the second position of the preset window;

[0165] Based on the second position of the preset window and the size of the preset window, multiple video stream slices corresponding to the target time slice are determined.

[0166] Optionally, the determining module is specifically used for:

[0167] Based on the priority of each first video stream, determine the second video stream with the highest priority and the slices of each second video stream;

[0168] The remaining amount of the preset window is determined based on the size of the preset window and the number of the second video stream with the highest priority.

[0169] Based on the remaining amount of the preset window, at least one third video stream is scheduled through the first round-robin method to determine at least one scheduled fourth video stream and the slices of each fourth video stream. The at least one third video stream is a video stream other than the second video stream with the highest priority among the multiple first video streams.

[0170] Based on the slices of each second video stream and each fourth video stream, multiple video stream slices corresponding to the target time slice are determined.

[0171] Optionally, before scheduling multiple first video streams based on a preset window and using a time-slice rotation method to determine the multiple video stream slices corresponding to the target time slice, the determining module is further configured to:

[0172] Determine whether the priorities corresponding to the multiple first video streams are the same;

[0173] If it is determined that the multiple first video streams correspond to the same priority, then the time slice rotation method is determined to be the first rotation method;

[0174] Alternatively, if it is determined that the priorities of the multiple first video streams are not the same, then the time slice rotation method is determined to be the second rotation method.

[0175] Optionally, before scheduling multiple first video streams based on a preset window and using a time-slice rotation method to determine the multiple video stream slices corresponding to the target time slice, the determining module is further configured to:

[0176] Based on the server's computing power and time slice length, determine the maximum number of video stream slices that the server can process within the time slice length.

[0177] Based on the maximum number of slices and a preset ratio value, the size of the preset window is determined, whereby the preset ratio value represents the proportional relationship between the size of the preset window and the maximum number of slices.

[0178] Optionally, before scheduling multiple first video streams based on a preset window and using a time-slice rotation method to determine the multiple video stream slices corresponding to the target time slice, the determining module is further configured to:

[0179] Obtain the operating status of each video stream device, wherein the operating status is either power-on or power-off;

[0180] Based on the operating status of each video streaming device, multiple target video streaming devices that are powered on are identified;

[0181] Based on the video streams corresponding to the multiple target video stream devices, the multiple first video streams are determined.

[0182] Figure 13 This is a second structural schematic diagram of the video stream processing device provided by the present invention, as shown below. Figure 13 As shown, the device includes a video stream acquisition platform 1301, a video stream scheduling and distribution platform 1302, and an algorithm scheduling platform 1303. The video stream scheduling and distribution platform 1302 includes the aforementioned determination module and the aforementioned interface module.

[0183] Specifically, the video stream acquisition platform, as the input end of the system, can acquire real-time video stream devices. The video stream scheduling and distribution platform can schedule video streams, accept all video streams in the northbound direction, schedule the video streams using its own scheduling algorithm, and transmit the video streams corresponding to the time slices to the algorithm scheduling platform for inference through the southbound interface. The algorithm scheduling platform can implement complex inference algorithms and push, display, and alarm the inference results.

[0184] Understandably, the addition of a video stream scheduling and distribution platform can decouple video streams from algorithm scheduling. The algorithm scheduling platform only needs to focus on the information exposed by the interface of the video stream scheduling and distribution platform. The stability of the video stream and its power-on / off status can be maintained by the video stream scheduling and distribution platform, which can prevent all algorithms from stopping inference due to the sudden loss of one or more cameras, thus improving the stability of server operation.

[0185] The video stream processing device provided by this invention schedules multiple first video streams using a time-slice rotation method. This allows for the determination of multiple video stream slices corresponding to a target time slice. These video stream slices are then input into an algorithm engine, which analyzes and infers the video stream slices to obtain inference results. Since the size of the preset window is determined based on the server's computing power, the total number of video stream slices is less than or equal to the size of the preset window. This ensures that the algorithm engine processes all the video stream slices within the target time slice, preventing system congestion due to insufficient decoding capabilities. This also ensures stable operation of the edge server even when many video stream devices are connected to it.

[0186] Figure 14 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 14As shown, the electronic device may include: a processor 1410, a communications interface 1420, a memory 1430, and a communication bus 1440, wherein the processor 1410, the communications interface 1420, and the memory 1430 communicate with each other via the communication bus 1440. The processor 1410 can call logical instructions in the memory 1430 to execute a video stream processing method, which includes:

[0187] Based on a preset window, multiple first video streams are scheduled using a time-slice rotation method to determine multiple video stream slices corresponding to a target time slice. The total number of the multiple video stream slices is less than or equal to the size of the preset window, which is determined based on the server's computing power performance.

[0188] Multiple video stream slices corresponding to the target time slice are input into the algorithm engine, which is used to analyze and reason about the video stream slices to obtain reasoning results.

[0189] Furthermore, the logical instructions in the aforementioned memory 1430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0190] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the video stream processing methods provided by the methods described above, the method comprising:

[0191] Based on a preset window, multiple first video streams are scheduled using a time-slice rotation method to determine multiple video stream slices corresponding to a target time slice. The total number of the multiple video stream slices is less than or equal to the size of the preset window, which is determined based on the server's computing power performance.

[0192] Multiple video stream slices corresponding to the target time slice are input into the algorithm engine, which is used to analyze and reason about the video stream slices to obtain reasoning results.

[0193] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0194] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0195] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video stream processing method, characterized in that, include: Based on a preset window, multiple first video streams are scheduled using a time-slice rotation method to determine multiple video stream slices corresponding to a target time slice. The total number of the multiple video stream slices is less than or equal to the size of the preset window, which is determined based on the server's computing power performance. Input multiple video stream slices corresponding to the target time slice into the algorithm engine. The algorithm engine is used to analyze and reason about the video stream slices to obtain reasoning results. When it is determined that the multiple video stream slices corresponding to the target time slice have been analyzed and reasoned, the next time slice is rotated for reasoning. Before scheduling multiple first video streams based on a preset window and using a time-slice rotation method to determine the multiple video stream slices corresponding to the target time slice, the method further includes: Based on the server's computing power and time slice length, determine the maximum number of video stream slices that the server can process within the time slice length. Based on the maximum number of slices and a preset ratio value, the size of the preset window is determined, whereby the preset ratio value represents the proportional relationship between the size of the preset window and the maximum number of slices.

2. The video stream processing method according to claim 1, characterized in that, The method of scheduling multiple first video streams based on a preset window and using a time-slice rotation to determine multiple video stream slices corresponding to a target time slice includes: When the time slice rotation mode is the first rotation mode, the multiple first video streams are scheduled based on a preset window using the first rotation mode to determine the multiple video stream slices corresponding to the target time slice; Alternatively, if the time slice rotation method is the second rotation method, the multiple first video streams are scheduled based on a preset window using the second rotation method to determine the multiple video stream slices corresponding to the target time slice; In the first round-robin mode, the scheduling probability of each video stream is equal; in the second round-robin mode, the scheduling probability of a higher-priority video stream is higher than that of a lower-priority video stream.

3. The video stream processing method according to claim 2, characterized in that, The step of scheduling the multiple first video streams based on a preset window and using a first round-robin method to determine the multiple video stream slices corresponding to the target time slice includes: Based on the sorting of the multiple first video streams, the first position of the preset window is determined; Based on the sliding step size, the preset window is slid according to the sorting of the multiple first video streams to determine the second position of the preset window; Based on the second position of the preset window and the size of the preset window, multiple video stream slices corresponding to the target time slice are determined.

4. The video stream processing method according to claim 2, characterized in that, The step of scheduling the multiple first video streams based on a preset window and using a second round-robin method to determine the multiple video stream slices corresponding to the target time slice includes: Based on the priority of each first video stream, determine the second video stream with the highest priority and the slices of each second video stream; The remaining amount of the preset window is determined based on the size of the preset window and the number of the second video stream with the highest priority. Based on the remaining amount of the preset window, at least one third video stream is scheduled through the first round-robin method to determine at least one scheduled fourth video stream and the slices of each fourth video stream. The at least one third video stream is a video stream other than the second video stream with the highest priority among the multiple first video streams. Based on the slices of each second video stream and each fourth video stream, multiple video stream slices corresponding to the target time slice are determined.

5. The video stream processing method according to claim 2, characterized in that, Before scheduling multiple first video streams based on a preset window and using a time-slice rotation method to determine the multiple video stream slices corresponding to the target time slice, the method further includes: Determine whether the priorities corresponding to the multiple first video streams are the same; If it is determined that the multiple first video streams correspond to the same priority, then the time slice rotation method is determined to be the first rotation method; Alternatively, if it is determined that the priorities of the multiple first video streams are not the same, then the time slice rotation method is determined to be the second rotation method.

6. The video stream processing method according to any one of claims 1-5, characterized in that, Before scheduling multiple first video streams based on a preset window and using a time-slice rotation method to determine the multiple video stream slices corresponding to the target time slice, the method further includes: Obtain the operating status of each video stream device, wherein the operating status is either power-on or power-off; Based on the operating status of each video streaming device, multiple target video streaming devices that are powered on are identified; Based on the video streams corresponding to the multiple target video stream devices, the multiple first video streams are determined.

7. A video stream processing device, characterized in that, include: The determination module is used to schedule multiple first video streams based on a preset window and in a time-slice rotation manner, and determine multiple video stream slices corresponding to a target time slice. The total number of the multiple video stream slices is less than or equal to the size of the preset window, and the size of the preset window is determined based on the computing power performance of the server. The interface module is used to input multiple video stream slices corresponding to the target time slice to the algorithm engine. The algorithm engine is used to analyze and reason about the video stream slices to obtain reasoning results. When it is determined that the multiple video stream slices corresponding to the target time slice have been analyzed and reasoned, the next time slice is rotated for reasoning. Before scheduling multiple first video streams based on a preset window and using a time-slice rotation method to determine the multiple video stream slices corresponding to the target time slice, the determining module is further configured to: Based on the server's computing power and time slice length, determine the maximum number of video stream slices that the server can process within the time slice length. Based on the maximum number of slices and a preset ratio value, the size of the preset window is determined, whereby the preset ratio value represents the proportional relationship between the size of the preset window and the maximum number of slices.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the video stream processing method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the video stream processing method as described in any one of claims 1 to 6.

Images