Automatic execution system and method for dynamically managing video stream identification task

Abstract

The invention discloses an automatic execution system and method for dynamically managing a video stream identification task, and relates to the technical field of video processing, and the system is characterized in that the system comprises a video stream input module, a model pool module, a load monitoring module, a scheduling module and an output module; the video stream input module is connected with the load monitoring module and the scheduling module, and the load monitoring module and the model pool module are connected with the scheduling module. The technical problem to be solved by the invention is to provide the automatic execution system and method for dynamically managing the video stream identification task, and by monitoring model load and video stream requirements in real time and dynamically adjusting video stream distribution and model multiplexing, system resources are utilized to the maximum extent, the processing efficiency is improved, and resource waste is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video processing, in particular to an automatic execution system and method for dynamically managing video stream identification tasks. BACKGROUND

[0002] Video stream processing systems analyze and identify video data through computer vision and deep learning techniques, and are widely used in security monitoring, intelligent transportation, industrial detection and other fields. With the complexity of deep learning models and the explosive growth of video stream data, traditional video stream processing solutions gradually expose problems such as low efficiency and resource waste. At present, video stream processing technology mainly relies on fixed allocation mode and static load balancing strategy. The system processes video streams in parallel through multi-thread or multi-process technology, but lacks dynamic adjustment capability.

[0003] Early video stream processing systems usually adopt fixed allocation mode, i.e. one-to-one binding of video streams and processing models, and the processing capacity of models cannot be dynamically adjusted according to real-time load. The disadvantage of this scheme is low resource utilization, and some models may be overloaded due to processing complex video streams, while other models are in idle state. In the 1990s, with the development of multi-core processors and distributed computing technology, static load balancing strategies (such as round robin, random allocation, etc.) were introduced into video stream processing systems, but these strategies still cannot adapt to the dynamic changes of video stream quantity and model load.

[0004] In the 21st century, with the rapid development of deep learning technology, video stream processing systems began to use GPU acceleration and distributed computing frameworks (such as TensorFlow, PyTorch, etc.) to improve processing efficiency. However, these technical solutions still have problems such as low model reuse rate and insufficient real-time performance. In recent years, with the popularity of edge computing and cloud computing technology, video stream processing tasks are allocated to cloud or edge devices for execution, but how to efficiently schedule resources is still a challenge. Existing scheduling algorithms are usually designed for static tasks and are difficult to adapt to the dynamic needs of video stream processing, resulting in poor system scalability and low energy efficiency.

[0005] Traditional video stream processing systems adopt fixed architecture and static scheduling strategy, which has the following disadvantages: low resource utilization, uneven model load, some models are overloaded while others are idle; low model reuse rate, unable to dynamically reuse models according to demand, leading to waste of computing resources; insufficient real-time performance, static scheduling strategy cannot adapt to the dynamic changes of video stream quantity and model load; poor system scalability: fixed architecture is difficult to dynamically expand or shrink resources according to business needs; low energy efficiency: resource waste leads to increased energy consumption, especially in edge computing environment.

[0006] The main task of a video stream processing system is to analyze and identify video streams in real time. However, the traditional video stream processing scheme is inefficient, with a resource utilization rate usually less than 50%, a model reuse rate less than 30%, and a lack of dynamic adjustment capability, resulting in high system response delay, low energy efficiency, and high hardware cost. Therefore, improving the efficiency, real-time performance, and resource utilization rate of the video stream processing system is a core problem to be solved. SUMMARY

[0007] The technical problem to be solved by the present application is to provide an automatic execution system and method for dynamically managing video stream identification tasks, which dynamically adjusts video stream distribution and model reuse by monitoring model load and video stream demand in real time, to maximize the use of system resources, improve processing efficiency, and reduce resource waste.

[0008] The present application achieves the intended purpose by adopting the following technical solutions: An automatic execution system for dynamically managing video stream identification tasks, characterized by comprising a video stream input module, a model pool module, a load monitoring module, a scheduling module, and an output module; the video stream input module is connected to the load monitoring module and the scheduling module, the load monitoring module and the model pool module are connected to the scheduling module, and the scheduling module is connected to the output module; the video stream input module receives video stream data from a camera or a video file; the model pool module contains multiple deep learning models for processing video stream data; the load monitoring module monitors the load of each model in real time, including processing speed, memory occupation, and GPU utilization; the scheduling module dynamically adjusts video stream distribution and model reuse according to monitoring data, adopts load balancing and model reuse strategies to ensure that the load of each model is in an optimal state; and the output module outputs the processed video stream data.

[0009] As a further limitation of the present technical solution, the scheduling module includes several model processing units, each of which includes several models.

[0010] As a further limitation of the present technical solution, the scheduling module further includes a protection mechanism, which includes a load threshold detector and a task re-distributor, and the models are connected to the corresponding load threshold detectors.

[0011] An execution method of an automatic execution system for dynamically managing video stream identification tasks, comprising the following steps: S1: The video stream input module receives a new video from a camera or a video file; S2: When a new video stream arrives, the scheduling module first checks the load of the current model in the current model processing unit. If the load of the current model is below the threshold, S3 is executed, otherwise S4 is executed; S3: assign the video stream to the model, perform S7; S4: if the load of all remaining models in the current model processing unit is below the threshold, perform S5, otherwise perform S6; S5: assign the video stream to the model with the lowest load among the remaining models, perform S7; S6: start a new model processing unit, perform S4-S6; S7: output the processing result.

[0012] As a further limitation of the technical solution, the deep learning model is a flame recognition model, which contains multiple judgments in addition to recognizing targets: Confidence filtering, determine whether it is an alarm: (1); Where: Ci represents the confidence score of the ith detection box; Ci represents the threshold value of the class corresponding to the ith detection box, and the calculation method is: (2); Where: Ci represents the class label of the ith detection box; Ci represents the class threshold mapping table; Ci represents the class corresponding threshold value; Ci represents the default threshold value; Alarm count logic, count the number of alarms: (3); Where: ; ; S j Ci represents the confidence score of the jth detection box; C j Ci represents the class label of the jth detection box; T(c) represents the confidence threshold of class c, defined as: (4); Frame area filtering, discard detection boxes with an area less than the threshold value; (5); Where: Ci represents the coordinates of the ith detection box; Ci represents the pixel area of the detection box; Sthresh excursion_size, which represents the frame bounding box area filtering threshold; frame bounding box overlap calculation, to calculate whether the frame bounding box is dynamic: (6) ; Wherein: M represents the number of detection boxes in the current frame; N represents the number of historical frames; L i represents the number of detection boxes in the i-th historical frame; B j coordinates of the j-th detection box in the current frame ; excursion_size, which represents the minimum detection box area threshold; ; ; T static (c) represents the static tracking threshold of category c; C min (c) represents the minimum overlap threshold of category c; IoU(B j ,B i,k ) represents the intersection over union calculation result of two detection boxes; S i,k represents the confidence of the historical detection box, which is used for filtering invalid detection; B i,k represents the coordinates of the historical detection box, which is used for spatial overlap calculation; final alarm condition calculation, whether to produce alarm: (7) ; Wherein: N min is the minimum alarm number required to trigger a real alarm.

[0013] As a further limitation of the technical solution, the load balancing algorithm formula: process allocation decision logic: For task , the task key is k, and the condition for assigning it to process is: (8) ; Wherein: S is a video stream; is a task set in process ; is the number of tasks; is the maximum number of tasks in a process; is the maximum number of streams in a process; is the maximum number of video streams in a process; is the maximum number of video streams in a process; is the current task associated video stream; Model-based process allocation: When the model of a task is , it is preferentially allocated to a process that already has this model: (9); wherein: is the set of processes containing the model ; If there is no process that meets the conditions, a new process is created: (10); Load balancing strategy: When a process has met , the following operations are performed: Stream balancing: if a process contains multiple streams, the old tasks of non-current task streams are preferentially replaced: (11); wherein: is any one task i in the process ; arg is the parameter corresponding to the task with the smallest timestamp; is the task timestamp; is the timestamp of the earliest created task in the process other than the current task; reassign to other processes; Process replacement: if it cannot be balanced, a new process is created and part of the tasks are migrated. As a further limitation of the technical solution, when multiple video streams require the same model, the scheduling module will assign them to the same model processing unit to improve the reuse rate.

[0014] As a further limitation of the technical solution, the processing of the video stream is as follows:

[0015] Stream state monitoring and restart logic For a video stream S, the state maintenance condition is: ​ (12); Wherein: Indicates the flow state; Indicates the continuous empty frame count; Indicates the current timestamp; Indicates the last picture taking time; Indicates that the cached picture is not expired; Indicates the automatic restart identifier; Peak clipping picture taking trigger rule: The trigger condition of picture taking operation Cap(S) is: (13); The picture naming rule is: (14); Wherein: t represents the current timestamp; Min(t) represents the minute; SC(S) represents the flow encoding; PE(S,t) represents that the picture already exists; Exe represents execution; RetC represents return cache; Indicates the year / month / day / hour; Frame chasing operation trigger condition; For RTSP / FLV type flow S, frame chasing operation Che(S) trigger is: (15); Wherein: S.PT represents the flow protocol type; Sec(t) represents the second; Exe(800) represents the number of frame chasing executions; landMin(t) represents the minute limit of frame chasing execution trigger condition; landSec(t) represents the second limit of frame chasing execution trigger time; Flow restart logic: Video flow S restart operation Res(S) execution condition is: (16); After restart, the state is updated to: ; Wherein: EC(S) represents the restart operation.

[0016] As a further limitation of the technical solution, the scheduling module is implemented by multi-process technology for parallel processing, and each process is responsible for processing a certain number of video stream tasks, and the processes communicate through shared memory.

[0017] Compared with the prior art, the advantages and positive effects of the present application are: 1. The present application dynamically adjusts video stream distribution and model reuse by real-time monitoring of model load and video stream demand through a dynamic scheduling algorithm, combined with load balancing and model reuse strategies, significantly improving resource utilization and processing efficiency. The protection points include the specific implementation method of the dynamic scheduling algorithm, the design of the load monitoring module, the implementation method of the model reuse technology, the implementation method of the multi-process processing, the design of the protection mechanism and the design of the system architecture.

[0018] 2. The present application dynamically adjusts video stream distribution and model reuse through a dynamic scheduling algorithm and a load monitoring module, significantly improving resource utilization and model reuse rate, and reducing resource waste. At the same time, the system has stronger real-time, scalability and dynamic adjustment capability, and can adapt to sudden traffic and dynamic demand. In addition, by optimizing the scheduling algorithm, the system energy consumption and hardware cost are reduced, which has a significant advantage in edge computing and embedded devices. The present application is superior to the traditional static scheduling scheme in performance, efficiency and cost. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a structural schematic diagram of the present application.

[0020] Figure 2 is a schematic diagram of the execution method of the present application.

[0021] Figure 3 is a schematic diagram of the multi-process technology parallel processing of the present application. DETAILED DESCRIPTION

[0022] The present application will be described in detail below with reference to the accompanying drawings, but it should be understood that the scope of protection of the present application is not limited by the specific embodiments.

[0023] The dynamic scheduling algorithm currently available has the characteristics of being able to dynamically adjust resource allocation according to real-time load, and the theoretical resource utilization rate can reach more than 90%, which can also significantly improve system performance in practical applications, so it is our preferred technical solution. In the dynamic scheduling algorithm, the scheduling strategy based on load balancing and model reuse is the key to efficient video stream processing. Among them, the load balancing algorithm can effectively distribute video stream tasks to avoid model overload or idling; the model reuse technology can maximize the use of model computing power and reduce resource waste.

[0024] In addition, the video stream processing system also needs to realize the basic functions of real-time monitoring, dynamic adjustment and scalability support. Therefore, it is also necessary to design real-time monitoring modules, dynamic adjustment modules and scalable architecture and other functional modules. The real-time monitoring module needs to realize the real-time monitoring of model load and video stream demand; the dynamic adjustment module needs to dynamically adjust the scheduling strategy according to the monitoring data; the scalable architecture needs to support the dynamic expansion or contraction of resources according to the business needs of the system.

[0025] Real-time monitoring and dynamic adjustment functions usually need to be implemented in combination with high-performance computing frameworks (such as TensorFlow, PyTorch, etc.) and distributed computing technologies (such as Kubernetes, Docker, etc.). This part of work involves complex technical implementation, but through the optimization of scheduling algorithms and system architecture, we can significantly improve the efficiency, real-time performance and scalability of the video stream processing system, thereby meeting the needs of large-scale video stream processing.

[0026] The core idea of the dynamic scheduling algorithm is to dynamically adjust the task allocation strategy through real-time monitoring of model load and video stream demand.

[0027] The application includes: a video stream input module, a model pool module, a load monitoring module, a scheduling module and an output module; the video stream input module is connected to the load monitoring module and the scheduling module, the load monitoring module and the model pool module are connected to the scheduling module, and the scheduling module is connected to the output module; the video stream input module receives video stream data from a camera or a video file; the model pool module contains multiple deep learning models for processing video stream data; the load monitoring module monitors the load of each model in real time, including processing speed, memory occupation and GPU utilization; the scheduling module dynamically adjusts video stream allocation and model reuse according to monitoring data, adopts load balancing and model reuse strategy to ensure that the load of each model is in the optimal state; the output module outputs the processed video stream data.

[0028] The specific implementation method of the dynamic adjustment algorithm for video stream is as follows: 1. Double-strategy load balancing: def distribution_model_task():# based on video stream affinity # Preferentially assign the same video stream task to the same process if stream_url in existing_streams: assign_to_existing_process() def distribution_stream_task():# based on model affinity # Try to process tasks of the same model together if model_name in model_process_map: reuse_model_process() 2、Resource allocation algorithm: # GPU allocation based on weighted round-robin next_gpu = (current_gpu + 1) % len(devices) os.environ["CUDA_VISIBLE_DEVICES"] = str(devices[next_gpu]) 3、Design of load monitoring module: def _get_gpu_load(self, device_id): """Get real-time GPU load (0-1 range)""" import pynvml pynvml.nvmlInit() handle = pynvml.nvmlDeviceGetHandleByIndex(device_id) util = pynvml.nvmlDeviceGetUtilizationRates(handle) mem_info = pynvml.nvmlDeviceGetMemoryInfo(handle) return { 'compute': util.gpu / 100.0, 'memory': mem_info.used / mem_info.total } def _calc_dynamic_weights(self): """Calculate dynamic weight factor""" weights = {} for d in self.devices: load = self._get_gpu_load(d) # Load penalty term (exponential decay) load_penalty = np.exp(-5*(1 - max(load['compute'], load['memory']))) # Historical stability factor (inverse of variance of last 10 loads) hist_loads = self.history[d][-10:]or [0.5] stability = 1 / (0.1 + np.var(hist_loads)) weights[d] = self.base_weights[d]* load_penalty * stability return weights Implementation of model reuse strategy technique: ```` class ModelPool: _instance = None _models = {}# Model cache pool def __new__(cls): if cls._instance is None: cls._instance = super().__new__(cls) return cls._instance @classmethod def get_model(cls, model_name, device): key = f"{model_name}_{device}" if key not in cls._models: cls._models[key] = cls._load_model(model_name, device) return cls._models[key] @classmethod def _load_model(cls, model_name, device): # Actual loading logic if model_name == "plate_detection": return load_plate_model(device) elif model_name == "fire_detection": return load_fire_model(device) #... other models Model loading optimization: def load_optimized_model(model_name, device): """Caching-aware model loading""" model = ModelPool.get_model(model_name, device) # GPU-specific optimization if str(device)!= 'cpu': model.enable_optimization() return model The scheduling module includes a plurality of model processing units, each of which includes a plurality of models.

[0029] The scheduling module further includes a protection mechanism, which includes a load threshold detector and a task re-distributor, and the models are connected to the corresponding load threshold detectors.

[0030] An execution method of an automatic execution system for dynamically managing video stream recognition tasks, comprising the following steps: S1: a video stream input module receives a new video from a camera or a video file; S2: when a new video stream arrives, the scheduling module first checks the load of the current model in the current model processing unit. If the load of the current model is below the threshold, execute S3, otherwise execute S4; S3: assign the video stream to the model, and execute S7; S4: when at least one of the remaining models in the current model processing unit has a load not exceeding the threshold, execute S5, otherwise execute S6; S5: assign the video stream to the model with the lowest load among the remaining models, and execute S7; S6: start a new model processing unit, and execute S4-S6; S7: output the processing result.

[0031] The deep learning model is a flame recognition model, which, in addition to identifying the target, also includes multiple judgments: Confidence filtering to determine whether it is an alarm: (1); in: This represents the confidence score of the i-th detection box; The threshold value for the category corresponding to the i-th detection box is calculated as follows: (2); in: This represents the category label of the i-th detection box; Represents the category threshold mapping table (score_threshold); Indicate category The corresponding threshold; This represents the default threshold (score_threshold["def"]). Alarm counting logic, to count the number of times an alarm occurs: (3); in: ; ; S j This represents the confidence score of the j-th detection box; C j This represents the category label of the j-th detection box; T(c) represents the confidence threshold for category c, defined as: (4); Frame area filtering discards detection frames with an area smaller than a threshold. (5); in: Represents the coordinates of the i-th detection box; Represents the pixel area of ​​the detection box; S thresh This represents the frame area filtering threshold (excursion_size in the code). Calculate the overlap of the frames and whether the frames are flexible: (6); Where: M represents the number of detection boxes in the current frame; N represents the number of historical frames; Li Number of bounding boxes of the i-th history frame B j Coordinates of the j-th bounding box of the current frame ; Minimum bounding box area threshold (excursion_size in code) ; ; T static (c) Static tracking threshold of class c C min (c) Minimum overlap threshold of class c IoU(B j ,B i,k ) represents the intersection over union calculation result of two bounding boxes S i,k Confidence of the history bounding box, used for filtering invalid detections B i,k Coordinates of the history bounding box, used for spatial overlap calculation Final alarm condition calculation, whether to generate an alarm (7) Where: N min Minimum number of alarms required to trigger a real alarm

[0032] Load balancing algorithm formula: Process allocation decision logic: For task , the task key is k, and the condition for assigning it to process is: (8) Where: S is the video stream is the task set in process ; is the number of tasks is the video stream set in the process is the number of streams is the maximum number of tasks per process is the maximum number of video streams per process is the video stream associated with the current task ​Model-based process allocation: When the task The model is When the time comes, priority will be given to processes that already have the model: (9); in: For inclusion model A collection of processes; If no matching process exists, a new process will be created: (10); Load balancing strategy: When process Satisfied Perform the following operations: Stream balancing: If a process contains multiple streams, prioritize replacing older tasks in streams that are not currently in use. (11); in: For process Any task i in the list is the task index; arg represents the task with the smallest timestamp. The corresponding parameters; For task timestamps; To obtain the timestamp of the earliest created (or earliest executed) task within the process, excluding the current task; Will Reassigned to other processes; Process replacement: If balancing cannot be achieved, create a new process and migrate some tasks.

[0033] When multiple video streams require the same model, the scheduling module will assign them to the same model processing unit to improve reuse rate.

[0034] The video stream is processed as follows: Stream status monitoring and restart logic For video stream S, the state maintenance condition is: (12); in: Indicates the stream status (1 = running, 3 = failed, 6 = restarting); Represents the empty frame count. Indicates the current timestamp; Last Take Time; Cache Image Expired; Auto Restart Flag (1=enable, 0=disable); Peak clipping take picture trigger rule: The trigger condition of take picture operation Cap(S) is: (13); Picture naming rule is: (14); Wherein: t represents the current time stamp; Min(t) represents the minute; SC(S) represents the stream code; PE(S,t) represents the picture exists; Exe represents the execution; RetC represents the return cache; Represents the year / month / day / hour; Frame chasing operation trigger condition; For RTSP (Real Time Streaming Protocol) / FLV (Flash Video) type stream S, the frame chasing operation Che(S) trigger is: (15); Wherein: S.PT represents the stream protocol type; Sec(t) represents the second; Exe(800) represents the frame chasing execution number; landMin(t) represents the minute limit of frame chasing execution trigger condition; landSec(t) represents the second limit of frame chasing execution trigger time; Stream restart logic: The execution condition of video stream S restart operation Res(S) is: (16); The state after restart is updated as: ; wherein: EC(S) represents a restart operation.

[0035] The scheduling module is implemented by a multi-process technique to realize parallel processing, each process is responsible for processing a certain number of video stream tasks, and the processes communicate through shared memory.

[0036] A cross-process shared state storage technique is used to guarantee state consistency among multiple processes, a process management module is separately arranged to be responsible for process state monitoring and recycling, a load balancing algorithm is used to realize maximum utilization and sharing of resources, and a performance optimization technique is used to optimize the GPU and CPU stream taking modes.

[0037] The above only discloses specific embodiments of the present application, but the present application is not limited thereto, and any changes that can be thought of by those skilled in the art shall fall within the protection scope of the present application.

Claims

1. A system for dynamically managing automated execution of video stream recognition tasks, the system comprising: The application relates to a video stream processing system, comprising a video stream input module, a model pool module, a load monitoring module, a scheduling module and an output module. The video stream input module is connected to the load monitoring module and the scheduling module, the load monitoring module and the model pool module are connected to the scheduling module, and the scheduling module is connected to the output module. The video stream input module receives video stream data from a camera or a video file. The model pool module comprises a plurality of deep learning models for processing video stream data. The load monitoring module monitors the load of each model in real time, including processing speed, memory occupation and GPU utilization. The scheduling module dynamically adjusts video stream distribution and model reuse according to monitoring data, adopts a load balancing and model reuse strategy, and ensures that the load of each model is in an optimal state. The output module outputs the processed video stream data. The scheduling module comprises a plurality of model processing units, and each model processing unit comprises a plurality of models.

2. The automated execution system for dynamic management of video stream identification tasks of claim 1, wherein: The scheduling module further comprises a protection mechanism, and the protection mechanism comprises a load threshold detector and a task re-distributor.

3. The automated execution system for dynamic management of video stream identification tasks of claim 2, wherein: The application further relates to a video stream processing method, comprising the following steps:

4. An execution method of an automated execution system for dynamically managing video stream identification tasks according to claim 3, characterized in that, S1: the video stream input module receives a new video from a camera or a video file; S2: when a new video stream arrives, the scheduling module first checks the load of the current model in the current model processing unit, and if the load of the current model is lower than a threshold value, S3 is executed, otherwise S4 is executed; S3: the video stream is distributed to the model, and S7 is executed; S4: when the load of at least one of the remaining models in the current model processing unit does not exceed the threshold value, S5 is executed, otherwise S6 is executed; S5: the video stream is distributed to the model with the lowest load among the remaining models, and S7 is executed; S6: a new model processing unit is started, and S4-S6 are executed; S7: the processing result is outputted. The deep learning model is a flame recognition model, and in addition to recognizing targets, the model further comprises a plurality of judgments, confidence filtering, alarm counting logic and T(c) representing the confidence threshold of category c, which is defined as follows: frame area filtering, frame body overlap calculation, final alarm condition calculation, load balancing algorithm formula, process allocation decision logic, model-based process allocation, if there is no process meeting the conditions, a new process is created, load balancing strategy, stream balancing, process replacement, when a plurality of video streams need the same model, the scheduling module distributes them to the same model processing unit to improve the reuse rate.

5. The execution method of claim 4, wherein: The processing of the video stream is as follows: Stream state monitoring and restart logic （1）； wherein: represents the confidence score of the ith detection box; Threshold value representing the category corresponding to the i-th detection frame, calculated as: （2）； wherein: represents the class label of the ith bounding box; representation category threshold mapping table; representative class corresponding threshold value denotes a default threshold value; For a video stream S, the state maintenance condition is as follows: （3）； wherein: ; ； S j a confidence score representing the jth detection box; C j a class label representing the jth detection box; Peak clipping and picture taking trigger rule （4）； The trigger condition of the picture taking operation Cap (S) is as follows: （5）； wherein: represents the coordinates of the ith detection frame; represents the pixel area of the detection frame; S thresh represents the frame area filtering threshold value; The picture naming rule is as follows: （6）； Wherein t represents the current timestamp, Min (t) represents the minute, and SC (S) represents the stream encoding. ​ L i represents the number of detection boxes of the i-th historical frame; B j coordinates of the jth detection frame of the current frame ; Minimum detection box area threshold; ； ； T static (c) a static tracking threshold value indicative of a class c; C min (c) a minimum overlap threshold value indicative of a class c; IoU(B j ,B i,k ) represents the intersection over union calculation result of two detection boxes; S i,k confidence of the history bounding box, used to filter invalid detections; B i,k Coordinates representing the historical bounding box for spatial overlap computation; ​ （7）； Where: N min Minimum number of alarms required to trigger a real alarm.

6. The execution method of claim 4, wherein: ​ ​ For task , the task key is k, and the condition for its assignment to the process is: （8）； ​ a set of tasks in a process a set of tasks in a process; n is the number of tasks; a set of video streams in progress; flow number; maxtasksperchild is the maximum number of tasks for a single process; MaxVideoStreamsPerProcess is the maximum number of video streams for a single process; is the current task associated video stream; ​ When a task of a model is , it is preferentially assigned to a process that already has the model: （9）； wherein: is a process set comprising a model ​ ​ （10）； ​ When the process has been full , the following operations are performed: ​ （11）； wherein: is a process is a task index; arg is the task that acquires the smallest timestamp corresponding parameters; task timestamp; To get the timestamp of the earliest created task within the process other than the current task; will be redistributed to other processes; reallocated to other processes; ​ 7. The execution method of claim 4, wherein: ​ 8. The execution method of claim 7, wherein: ​ ​ ​ （12）； wherein: represents the flow state; represents a consecutive null frame count; represents the current timestamp; represents the last time the picture was taken; CacheMapNotExpired indicates that the cache map is not expired; auto restart flag indicates an automatic restart; ​ ​ （13）； ​ （14）； ​ ​ ​ PE(S, t) represents that the picture exists; Exe represents execution; RetC represents return cache; represents year / month / day / hour; Frame chasing operation trigger condition; For RTSP / FLV type stream S, frame chasing operation Che(S) trigger is: （15）； Wherein: S.PT represents stream protocol type; Sec(t) represents seconds; Exe(800) represents the number of frame chasing execution times; landMin(t) represents the minute limit of frame chasing execution trigger condition; landSec(t) represents the second limit of frame chasing execution trigger time; Stream restart logic: Video stream S restart operation Res(S) execution condition is: （16）； The state after restart is updated as: ； Wherein: EC(S) represents restart operation.

9. The execution method of claim 4, wherein: The scheduling module is realized by multi-process technology and parallel processing, and each process is responsible for processing a certain number of video stream tasks, and the processes communicate through shared memory.

Images