Distributed video recording task scheduling method, device and system

Abstract

The invention discloses a distributed video recording task scheduling method, device and system, and provides a distributed video recording task distribution scheduling scheme based on a message queue, a video recording task management server obtains a video recording task needing to be distributed from a database, stores the obtained video recording task into a set message queue, and sends the message queue to a video recording task management server; according to the scheme provided by the invention, the distribution of the video recording task is dominated by each video recording node according to the local task execution condition, so that the distribution of the video recording task is more accordant with the characteristics of the actual distributed video recording task; the technical problem of poor system stability caused by centralized distribution of existing servers is solved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a distributed video recording task method, device and system. Background Art

[0002] In today's digital age, video surveillance systems have become an important tool for ensuring public safety and maintaining social order. With the development of Internet of Things technology, the number of video surveillance devices has exploded, which not only generates massive amounts of video data, but also places higher demands on the storage and management of video data.

[0003] Due to the special task characteristics of the recording task, such as the need for continuous execution, the need to continuously track the execution status, and the large difference in the completion time of a single task, the existing distributed task distribution logic cannot achieve scheduling that conforms to the actual characteristics of the distributed recording task, which is prone to node load imbalance and reduced recording completeness, which is not conducive to the stable operation of the system. Summary of the invention

[0004] The present application provides a distributed video recording task scheduling method, device and system for realizing the invention purpose of a distributed video recording task distribution management technology specialized for video recording tasks.

[0005] To achieve the above-mentioned invention object, the first aspect of the present application provides a distributed video recording task scheduling method, which is applied to a video recording task management server, comprising:

[0006] Initialize the message queue;

[0007] The recording tasks to be distributed are obtained from the database, and then the recording tasks are distributed to the message queue, so that each of the recording nodes can obtain the recording tasks from the message queue and execute the obtained recording tasks according to the local task execution status and task statistical data, combined with the preset message consumption logic.

[0008] Preferably, the initialization message queue includes:

[0009] According to the number of online recording nodes, combined with the preset relationship between the number of recording nodes and message queues, the number of queues of the message queue is determined, so as to generate a corresponding message queue according to the number of queues.

[0010] Preferably, the video recording task management server is a server cluster composed of multiple server nodes.

[0011] Preferably, when there are multiple message queues, distributing the video recording task to the message queues specifically includes:

[0012] Through the task polling method, the target message queue corresponding to each video recording task to be distributed is determined, so as to distribute each video recording task to the corresponding target message queue.

[0013] Preferably, it also includes:

[0014] receiving a feedback message sent by the recording node, and determining the operating status of the recording node according to the feedback message;

[0015] When a faulty recording node is detected, the unfinished recording tasks assigned to the faulty recording node are determined according to the historical records of tasks acquired by each recording node;

[0016] The unfinished video recording task is obtained from the database and republished to the message queue so as to be obtained and executed by the remaining online video recording nodes.

[0017] At the same time, the second aspect of the present application also provides a distributed video recording task scheduling method, which is applied to a video recording node, including:

[0018] Monitor the current task execution status and task statistics of the recording node;

[0019] When it is detected that the recording node is not in the task execution state and the task statistical data meets the preset task acquisition trigger condition, the recording task is obtained from the message queue according to the preset message consumption logic, and the obtained recording task is executed, wherein the message queue is generated according to the instructions of the recording task management server, and the recording task is obtained from the database by the recording task management server and distributed to the message queue.

[0020] Preferably, when there are multiple message queues, obtaining the video recording task from the message queue according to the preset message consumption logic includes:

[0021] According to the key-value characters and the number of online video recording nodes obtained in real time, a calculated value is obtained according to a preset consistent hash algorithm, so as to determine a target message queue according to the calculated value, wherein the key-value characters are generated based on the identification identifier of the video recording node and the current timestamp;

[0022] A video recording task is obtained from the target message queue, and the obtained video recording task is executed.

[0023] The third aspect of the present application further provides a video recording task management server device, including:

[0024] Message queue configuration unit, used to initialize the message queue;

[0025] The video recording task distribution unit is used to obtain the video recording tasks to be distributed from the database, and then distribute the video recording tasks to the message queue, so that each of the video recording nodes can obtain the video recording tasks from the message queue and execute the obtained video recording tasks based on the local task execution status and task statistical data, combined with the preset message consumption logic.

[0026] The fourth aspect of the present application further provides a recording node device, including:

[0027] A node status monitoring unit, used to monitor the current task execution status and task statistics of the recording node;

[0028] A task acquisition execution unit is used to obtain the recording task from the message queue according to the preset message consumption logic and execute the obtained recording task when it is detected that the recording node is not in the task execution state and the task statistical data meets the preset task acquisition trigger condition, wherein the message queue is generated according to the instructions of the recording task management server, and the recording task is obtained from the database by the recording task management server and distributed to the message queue.

[0029] The fifth aspect of the present application also provides a distributed video recording task scheduling system, including: a database, a message queue, a video recording task management server device as described in the third aspect of the present application, and a video recording node device as described in the fourth aspect of the present application.

[0030] It can be seen from the above technical solutions that this application has the following advantages:

[0031] The present application provides a distributed recording task distribution and scheduling scheme based on a message queue. The recording task management server obtains the recording tasks to be distributed from a database, and stores the obtained recording tasks in a set message queue for each recording node that meets the task acquisition conditions to obtain and execute. According to the scheme of the present application, the allocation of recording tasks is dominated by each recording node according to the local task execution status, which is more in line with the characteristics of actual distributed recording tasks and solves the technical problem of poor system stability caused by the existing centralized distribution of the server. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0033] Figure 1A schematic diagram of the architecture of an embodiment of a distributed video recording task scheduling system provided in this application.

[0034] Figure 2 A flow chart of an embodiment of a distributed video recording task scheduling method provided in this application.

[0035] Figure 3 This is a flowchart of an overall embodiment of a distributed task distribution process provided by the present application based on the above basic embodiment.

[0036] Figure 4 This is a flowchart of an overall embodiment for task status tracking provided by the present application based on the above basic embodiment.

[0037] Figure 5 A schematic diagram of the structure of a video recording task management server device in a distributed video recording task scheduling system provided in this application.

[0038] Figure 6 A schematic diagram of the structure of a video recording node device in a distributed video recording task scheduling system provided in this application. DETAILED DESCRIPTION

[0039] In view of the problems existing in the prior art, research has found that the existing distributed task distribution mechanism is mostly dominated by the server, that is, the server determines whether to issue distributed tasks and how to distribute tasks based on the task status of each execution terminal under it, and the judgment criteria commonly used in the existing method are mostly the local task volume reported by each execution terminal. However, since the recording task needs to continuously execute the recording action and track the execution status, the duration can be as short as a few minutes or as long as several days or even weeks, and the task time varies greatly. In the scenario of distributed recording task distribution, if the task volume of the recording node is low but the task is executed for a long time, according to the existing task allocation mechanism, the following situations may occur: the original task is interrupted by the newly assigned task, resulting in a decrease in the completeness rate of the recording, or the newly assigned recording task cannot be executed in a short time, causing the recording task to be delayed, or even the node load is unbalanced due to the long-term lag of the task.

[0040] In view of this, the present application provides a distributed video recording task scheduling method, device and system for achieving the invention purpose of a distributed video recording task distribution management technology specialized for video recording tasks.

[0041] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described below are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0042] Terminology explanation:

[0043] NRU: Recording node, the module that actually performs the recording task

[0044] NRM: Video recording task management server. Multiple NRMs form a cluster through a preset protocol.

[0045] Message queue: A message queue is a software system component that allows the sender (producer) of a message to send a message to a queue, and the receiver (consumer) of the message can retrieve and process the message from the queue. This mechanism eliminates the need for producers and consumers to communicate directly or be online at the same time, thus achieving asynchronous communication and decoupling.

[0046] Raft algorithm: Raft algorithm is a distributed consensus algorithm designed to simplify understanding and achieving consensus problems in distributed systems. In a distributed system, multiple nodes need to agree on a value, even in the event of a network partition or node failure. Raft algorithm simplifies this process by electing a leader to centrally handle all write operations.

[0047] Jump consistent hash: A consistent hashing algorithm that has the advantages of small memory usage and high speed. It only has about 5 lines of code and is more suitable for use in sharded distributed storage systems.

[0048] Nacos: A service discovery and configuration management system designed to help developers more easily manage and coordinate service instances in a microservice architecture. Both NRM nodes and NRU nodes can carry their own IDs to connect to the nacos cluster for service registration and enable the health detection function. After registration, they can be put into use.

[0049] First, a detailed description of an embodiment of a distributed video recording task scheduling system provided by the present application is as follows:

[0050] like Figure 1 As shown, the distributed video recording task scheduling system provided in this embodiment includes: a database, a video recording task management server NRM, a message queue and a video recording node NRU;

[0051] The video recording task is saved in the database, and the task status is set to undistributed. The specific information of a video recording task includes but is not limited to the task id, video channel id, id of the NRU that executes the task, task creation time, task start time, task end time, video stream parameters, task status, task status update timestamp, execution error information, etc. It should be noted that the task id and video channel id together constitute the unique key of the video recording task table; the task status is represented by an integer number, and the number corresponds to the specific meaning as follows: 0-undistributed, 1-acquired, 2-distributed, 3-in execution, 4-completed, and the id of the NRU that executes the task can be determined based on the information fed back after the task is distributed.

[0052] NRM will obtain a batch of undistributed tasks from the database at regular intervals or when other judgment conditions are met (for example, the task volume of the message queue is lower than a certain level), and set the status of these tasks to obtained, and then send the obtained tasks to the message queue. The distributed video task scheduling method on the NRM side can be found in the following specific embodiment.

[0053] The message queue is the middleware for transmitting the video task between NRM and NRU, realizing the asynchronous transmission of the video task;

[0054] NRU nodes are connected to the message queue, nacos cluster and NRM cluster respectively. There is no network connection between NRU nodes. After NRU is registered and connected to NRM through nacos, it will obtain the recording task from the message queue in combination with the preset message consumption logic when the conditions for triggering the task are met based on the local task execution status and task statistics. Then, the obtained recording task will be executed.

[0055] It should be noted that in order to further improve the high concurrency, high availability, high task execution rate and system stability of the entire task management system. The NRM, NRU, database, message queue, nacos, etc. mentioned in this embodiment can be configured in cluster form to improve the efficiency of data concurrent processing and avoid single node downtime causing errors in the overall operation of the system.

[0056] When NRM is a cluster, only NRM Leader will connect to the nacos cluster and the message queue, and is responsible for taking the task list from the database and distributing it to the queue of the message queue. The remaining NRM Followers only serve as backup nodes for NRM Leader, and receive and forward feedback messages sent by each NRU.

[0057] The above is an overall description of a distributed video recording task scheduling system provided by the present application. The following is a detailed description of an embodiment of a distributed video recording task scheduling method provided by the present application on the video recording task management server NRM side and the video recording node NRU side.

[0058] See also Figure 2 In a first aspect, the present application provides a distributed video recording task scheduling method, which is applied to a video recording task management server, comprising:

[0059] Step 101, initialize the message queue;

[0060] It should be noted that the NRM first issues an instruction to create a message queue. The configuration of the message queue includes:

[0061] According to the number of online recording nodes and the preset quantitative relationship between recording nodes and message queues, the number of message queues is determined to generate corresponding message queues according to the number of queues. For the specific implementation method, please refer to the following example: NRM can obtain the current online NRU list and the total number of NRU nodes N from nacos, and according to the number of online recording nodes and the preset quantitative relationship between recording nodes and message queues, determine the number of message queues, and then generate a corresponding number of message queues. Among them, this embodiment mentions that the quantitative relationship between recording nodes and message queues is preferably 3:1. If the number of recording nodes N<3, N / 3 is 1. If N is not divisible by 3, N / 3 is rounded up. The above quantitative relationship ratio is exemplary data. The actual ratio can be adjusted according to actual needs and is not limited here.

[0062] Step 102: Obtain the video recording task to be distributed from the database, and then distribute the video recording task to the message queue;

[0063] It should be noted that in the process of NRM obtaining tasks from the database, in addition to the method of obtaining tasks according to task status mentioned in the system embodiment, the task start time factor can be further considered, and undistributed recording tasks with a start time >= current time - t can be obtained first to ensure that tasks close to the start time can be distributed first, where t is a preset distribution buffer time, and the value range is generally 1 to 5 minutes.

[0064] After NRM obtains the corresponding recording task, it can distribute the recording task to the message queue to wait for each recording node to consume it. Each recording node can obtain the recording task from the message queue and execute the obtained recording task based on the local task execution status and task statistics data, combined with the preset message consumption logic.

[0065] It should be noted that, when the NRM is a server cluster composed of multiple server nodes, this embodiment also provides:

[0066] Through the preset election algorithm, the only master server node NRMLeader is determined from each server node, and the remaining server nodes serve as slave server nodes NRM Follower. Among them, the master server node is the executor of the steps of acquiring and distributing video tasks from the database to the message queue and monitoring the task status. The remaining slave server nodes can be used as backups for the master server node. Each NRU can communicate with any server node in the server cluster through the gRPC service protocol. If the NRU is connected to a slave server node, the slave server node will summarize the received information to the master server node, and the master server node will coordinate the task distribution.

[0067] The preset election algorithm mentioned in this embodiment may preferably adopt the Raft algorithm, which ensures that there is only one Leader responsible for distributing tasks in the NRM cluster to avoid repeated distribution of tasks. In order to ensure the effect of the algorithm, the number of initial NRM nodes needs to be an odd number >= 3. Other commonly used selection logics may also be used, for example, randomly selecting one from the registered and normally running server nodes as the main server node, or selecting one with the best hardware configuration / device performance as the main server node, etc., which are not listed one by one here.

[0068] When there are multiple message queues, the scheduling method on the NRM side distributes the recording task to the message queue specifically including:

[0069] Through the task polling method, the target message queue corresponding to each video recording task to be distributed is determined, so as to distribute each video recording task to the corresponding target message queue.

[0070] It should be noted that NRM Leader distributes the acquired tasks to queues numbered 1 to N / 3 in the message queue (N is the total number of nodes in the NRU cluster), and sets the status of the corresponding tasks in the database to distributed.

[0071] Furthermore, the scheduling method on the server side may also include:

[0072] Step 105: receiving a feedback message sent by the recording node, and determining the running status of the recording node according to the feedback message;

[0073] Step 106: When a faulty recording node is detected, the unfinished recording tasks assigned to the faulty recording node are determined according to the historical records of the tasks acquired by each recording node, so as to republish the unfinished recording tasks to the message queue.

[0074] It should be noted that each NRU will send feedback messages to the NRM periodically or after completing a task during operation. These feedback messages include but are not limited to: task execution status feedback messages and device activity feedback messages. If the NRM does not receive feedback messages sent by the NRU for a certain period of time, it can be determined that the NRU has an abnormal operation and cannot continue to execute the assigned task. At this time, the NRM can search the database for tasks associated with this NRU ID and whose status is not "completed" based on the NRU ID, and set the status of these tasks to "undistributed" so that they can be re-sent to the message queue and obtained and executed by other normal NRU devices.

[0075] like Figure 2 As shown, the present application also provides a distributed video recording task scheduling method, which is applied to a video recording node and includes:

[0076] Step 103: monitor the current task execution status and task statistics of the video recording node;

[0077] Step 104: when it is detected that the recording node is not in the task execution state and the task statistical data meets the preset task acquisition trigger condition, the recording task is obtained from the message queue according to the preset message consumption logic, and the obtained recording task is executed;

[0078] The message queue is generated according to the instruction of the video recording task management server, and the video recording task is obtained from the database by the video recording task management server and distributed to the message queue.

[0079] It should be noted that, first, each NRU detects the current local task execution status and task statistics, where the task statistics include but are not limited to: the remaining local task inventory, the number of tasks executed since the last task acquisition, etc.;

[0080] When the NRU is not in the task execution state and the task statistics meet the preset task acquisition trigger conditions, it will obtain the recording task from the message queue according to the preset message consumption logic and execute the acquired recording task. At the same time, the NRU will report feedback information to the NRM regularly or after the task is completed, so that the NRM can determine the active status of the NRU based on the received feedback information, and can also update the task status locally and in the database based on the task execution status feedback message in the feedback information.

[0081] Furthermore, when there are multiple message queues, the process of obtaining the video recording task from the message queue according to the preset message consumption logic in step 104 includes:

[0082] According to the key value characters and the number of online video recording nodes obtained in real time, a calculated value is obtained according to a preset consistent hash algorithm, so as to determine the target message queue according to the calculated value;

[0083] Among them, the key value characters mentioned in this embodiment are generated based on the identification mark of the recording node and the current timestamp. The specific generation method includes: the concatenated characters formed by concatenating the id of the recording node NRU and the current timestamp or the encrypted characters encrypted by encryption algorithms such as MD5.

[0084] For example, NRU obtains the total number of nodes N from the NRM cluster, and uses jump consistent hash to perform hash calculation based on the generated key value character and N. The jump consistent hash algorithm requires two integer numbers to be input, namely key and bucketnum. The key is the string obtained by md5 (NRU's id + current timestamp) converted to a decimal number, and bucketnum is the total number of nodes N. After algorithm processing, a number M in the range of [1, N] can be obtained, and it is connected to the queue numbered M / 3 for consumption; every period of time, NRU will repeat this step to recalculate M and consume. In addition to the above-mentioned jump consistent hash, other consistent hash algorithms such as Consistent Hashing or HRW Hashing can also be used.

[0085] The above are the step embodiments of a distributed video recording task scheduling method provided by the present application on the NRM side and the NRU side respectively. In order to more clearly explain the technical solution of the present application, the present application also provides the following embodiments for illustration.

[0086] See also Figure 3 , Figure 3 This is a flow chart of an overall embodiment of a distributed task distribution process provided by this application based on the above basic embodiment. The flow chart is as follows:

[0087] S1.1 Save the video recording task in the database and set the task status to undistributed. The specific information of a video recording task includes but is not limited to the task ID, video channel ID, ID of the NRU that executes the task, task creation time, task start time, task end time, video stream parameters, task status, task status update timestamp, execution error information, etc.

[0088] S1.2 NRM elects a leader through the raft algorithm and forms a cluster. Next, this leader is called NRM Leader, and the other NRMs that are not leaders are called NRM Followers. All nodes in NRM carry their own ID and gRPC service URL to connect to the nacos cluster for service registration and enable the health detection function.

[0089] The initial number of NRM nodes must be an odd number >= 3.

[0090] S1.3 All NRU nodes in the NRU cluster connect to nacos, register services with their own IDs, and enable the health detection function;

[0091] S1.4 The nodes in the NRU cluster randomly obtain an NRM node from nacos and use the gRPC protocol to connect to the gRPC service URL. The gRPC service URL is used by NRU to regularly upload task execution status to NRM.

[0092] S1.5 NRM Leader obtains the current online NRU list and the total number of NRU nodes N from nacos. If the total number of NRU nodes N(j+1) currently obtained is less than the N(j) obtained last time, delete the node numbered The queue within the range; because the NRM cluster is connected using the raft algorithm, the cluster itself is equivalent to a kv storage, and the number of the latest NRU nodes stored in the NRM cluster is N(j);

[0093] S1.6 NRM Leader periodically obtains a batch of tasks with a start time >= current time - 2 minutes and a status of undistributed from the database every minute, and sets the status of these tasks to obtained;

[0094] The number of tasks in each batch can be customized, and can generally be set to Nx100, where N is the number of NRU nodes;

[0095] S1.7 NRU obtains the total number of nodes N from the NRM cluster, and uses jump consistent hash to calculate the hash based on md5 (its own id + current timestamp) and N, and obtains a number M in the range [1, N], and connects to the queue numbered M for consumption; every once in a while, NRU will repeat this step to recalculate M and consume;

[0096] The jump consistent hash algorithm requires two integers, key and bucketnum. The key is the string obtained by md5 (NRU id + current timestamp) converted to a decimal number, and bucketnum is the total number of nodes N.

[0097] S1.8 NRM Leader polls and distributes the acquired tasks to queues numbered 1 to N / 3 in the message queue (N is the total number of nodes in the NRU cluster. If N<3, N / 3 is 1), and sets the status of the corresponding tasks in the database to distributed;

[0098] S1.9 According to the method in S1.7, NRU subscribes to the queue numbered M, obtains a batch of tasks and executes them, and immediately sends a task status update request to the gRPC URL of the NRM node to which it is connected, carrying the NRU id, task id, video channel id, and task status field (the value is executing);

[0099] The maximum number of tasks that NRU acquires each time is determined by its own hardware conditions (bandwidth, CPU, memory, etc.). In experience, when the video streams are all h.264 main streams with a resolution of 1080p, this value is generally 500~1000, which is named MAXTASK; NRU executes tasks in a concurrent manner. Since the execution time of each video task is different, after executing S tasks, NRU will try to obtain the next batch of tasks again and keep the current number of executed tasks <= MAXTASK. It can also trigger the action of NRU trying to obtain the next batch of tasks again when it detects that the number of remaining tasks of this NRU is lower than the threshold L.

[0100] S1.10 After receiving the task status update request, the NRM node updates the NRU id and task status into the database, sets the status in the database to "executing", and updates the status timestamp.

[0101] In addition, the present application also provides an overall embodiment of the task status tracking process based on the above basic embodiment, which is as follows:

[0102] See also Figure 4 , Figure 4 This is a flow chart of an overall embodiment for task status tracking provided by this application based on the above basic embodiment. The flow chart is as follows:

[0103] S2.1. During the execution of the task, NRU reports the current status of the task to the gRPC URL of NRM every K minutes, including the NRU id, task id, video channel id, task status, and error information. The task status is "executing" and the error information is the error that occurred during the task execution. NRM updates the id, task id, video channel id, task status, and current timestamp of the NRU executing the task to the database.

[0104] In order to reduce the possible loss of video recording time due to node downtime, the optimal range of K value is less than 5 minutes.

[0105] S2.2. After NRU completes the task, it immediately reports the current status of the task to the gRPC URL of NRM, including the ID of the NRU that executes the task, the task ID, the video channel ID, and the task status. The task status is "completed".

[0106] Among them, as for the processing flow after NRU crashes, because NRU and NRM use a heartbeat keep-alive mechanism, after NRM senses that the NRU connected to itself has been disconnected, it can confirm that the NRU cannot continue to perform the assigned tasks, so it carries the id of the NRU to notify the NRM Leader that the NRU has been offline. After receiving the request, the NRM Leader immediately goes to the database to find tasks associated with this NRUid and whose status is not "completed", and sets the status of these tasks to "undistributed".

[0107] S2.3, NRM Leader can also open an additional separate thread to scan the database every certain period of time (less than K minutes), and set the status of all tasks whose current time-status update timestamp is greater than K minutes and whose task status is not equal to "completed" to "undistributed";

[0108] Compared with existing solutions, the method proposed in this application specializes the characteristics of recording tasks. First, this application provides a distributed recording task distribution and scheduling scheme based on a message queue. In this scheduling scheme, the server NRM is only responsible for obtaining recording tasks from the database and sending the tasks to the message queue. Then, each recording node NRU obtains the recording tasks to be executed from the message queue based on its own operating conditions and when new tasks can be obtained. This can better improve the recording completeness rate and system stability, and avoids the defects of the traditional server-dominated distributed task sending mechanism in the face of distributed recording task management scenarios.

[0109] Secondly, the NRM, NRU, database, message queue, nacos, etc. in the present application method all adopt clusters to avoid system unavailability caused by single node failure, ensuring high concurrency, high availability and high task execution rate of the entire task management system, while taking into account the load balancing of NRU nodes;

[0110] At the same time, a dynamically adjusted recording task distribution mechanism is provided: the polling algorithm is used to ensure that tasks are evenly distributed to each queue of the message queue. Because multiple nodes are connected to a queue at the same time, it avoids the situation where a single node fails and causes no consumers to consume the queue. Among them, NRU will periodically recalculate M and change the queue it consumes.

[0111] If the number of NRU nodes decreases significantly due to capacity reduction, NRM Leader will delete the redundant queues in time to avoid the task data in the queues being consumed by consumers; if the number of NRU nodes increases significantly due to capacity expansion, NRM Leader will dynamically create new queues to ensure that tasks can be evenly distributed; this mechanism can adapt to the dynamic expansion and contraction of the system;

[0112] There is also a recovery mechanism for the unfinished recording tasks of abnormal NRUs: the NRU node maintains a TCP connection with the NRM cluster and has a keep-alive mechanism. Once the NRU goes down, the NRM can promptly sense and recover the tasks that the NRU is executing and redistribute them; among them, the NRM Leader can also regularly scan the unfinished tasks that have not been updated for a period of time and redistribute these tasks.

[0113] The above is a detailed description of an embodiment of a distributed video recording task scheduling method provided by the present application. The following is a detailed description of embodiments of a video recording task management server device and a video recording node device used for the above-mentioned distributed video recording task scheduling method provided by the present application.

[0114] See also Figure 5 The present application provides a video recording task management server device embodiment, characterized in that it includes:

[0115] The message queue configuration unit 201 is used to initialize the message queue;

[0116] The video recording task distribution unit 202 is used to obtain the video recording tasks to be distributed from the database, and then distribute the video recording tasks to the message queue, so that each video recording node can obtain the video recording tasks from the message queue and execute the obtained video recording tasks based on the local task execution status and task statistical data, combined with the preset message consumption logic.

[0117] See also Figure 6 The present application provides a video recording node device embodiment, characterized in that it includes:

[0118] The node status monitoring unit 203 is used to monitor the current task execution status and task statistics of the video recording node;

[0119] The task acquisition execution unit 204 is used to obtain the recording task from the message queue according to the preset message consumption logic and execute the obtained recording task when it is detected that the recording node is not in the task execution state and the task statistical data meets the preset task acquisition trigger condition, wherein the message queue is generated according to the instructions of the recording task management server, and the recording task is obtained from the database by the recording task management server and distributed to the message queue.

[0120] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the terminal equipment, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0121] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0122] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0123] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0124] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0125] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0126] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0127] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A distributed video recording task scheduling method, applied to a video recording task management server, characterized in that: include: Initialize the message queue; The recording tasks to be distributed are obtained from the database, and then the recording tasks are distributed to the message queue, so that each of the recording nodes can obtain the recording tasks from the message queue and execute the obtained recording tasks according to the local task execution status and task statistical data, combined with the preset message consumption logic.

2. A distributed video recording task scheduling method according to claim 1, characterized in that: The initialization message queue includes: According to the number of online recording nodes, combined with the preset relationship between the number of recording nodes and message queues, the number of queues of the message queue is determined, so as to generate a corresponding message queue according to the number of queues.

3. A distributed video recording task scheduling method according to claim 1, characterized in that: The video recording task management server is a server cluster composed of multiple server nodes.

4. A distributed video recording task scheduling method according to claim 3, characterized in that: When there are multiple message queues, distributing the video recording task to the message queues specifically includes: Through the task polling method, the target message queue corresponding to each video recording task to be distributed is determined, so as to distribute each video recording task to the corresponding target message queue.

5. A distributed video recording task scheduling method according to claim 1, characterized in that: Also includes: receiving a feedback message sent by the recording node, and determining the operating status of the recording node according to the feedback message; When a faulty recording node is detected, the unfinished recording tasks assigned to the faulty recording node are determined according to the historical records of tasks acquired by each recording node; The unfinished video recording task is obtained from the database and republished to the message queue so as to be obtained and executed by the remaining online video recording nodes.

6. A distributed video recording task scheduling method, applied to a video recording node, characterized in that: include: Monitor the current task execution status and task statistics of the recording node; When it is detected that the recording node is not in the task execution state and the task statistical data meets the preset task acquisition trigger condition, the recording task is obtained from the message queue according to the preset message consumption logic, and the obtained recording task is executed, wherein the message queue is generated according to the instructions of the recording task management server, and the recording task is obtained from the database by the recording task management server and distributed to the message queue.

7. A distributed video recording task scheduling method according to claim 6, characterized in that: When there are multiple message queues, obtaining the video recording task from the message queue according to the preset message consumption logic includes: According to the key-value characters and the number of online video recording nodes obtained in real time, a calculated value is obtained according to a preset consistent hash algorithm, so as to determine a target message queue according to the calculated value, wherein the key-value characters are generated based on the identification identifier of the video recording node and the current timestamp; A video recording task is obtained from the target message queue, and the obtained video recording task is executed.

8. A video recording task management server device, characterized in that: include: Message queue configuration unit, used to initialize the message queue; The video recording task distribution unit is used to obtain the video recording tasks to be distributed from the database, and then distribute the video recording tasks to the message queue, so that each of the video recording nodes can obtain the video recording tasks from the message queue and execute the obtained video recording tasks based on the local task execution status and task statistical data, combined with the preset message consumption logic.

9. A video recording node device, characterized in that: include: A node status monitoring unit, used to monitor the current task execution status and task statistics of the recording node; A task acquisition execution unit is used to obtain the recording task from the message queue according to the preset message consumption logic and execute the obtained recording task when it is detected that the recording node is not in the task execution state and the task statistical data meets the preset task acquisition trigger condition, wherein the message queue is generated according to the instructions of the recording task management server, and the recording task is obtained from the database by the recording task management server and distributed to the message queue.

10. A distributed video recording task scheduling system, characterized in that: include: A database, a message queue, a video recording task management server device as claimed in claim 8, and a video recording node device as claimed in claim 9.

Images