Container-based scalable distributed dual-queue dynamic allocation method in edge scenarios
By adopting collaborative computing methods and container technology of multiple edge servers in edge computing scenarios, efficient processing of video streams is achieved, the problem of insufficient computing power of a single edge server is solved, resource utilization and processing efficiency are improved, and the system supports Rapidly expand and upgrade.
Patent Information
- Application Number
- CN202011295965.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-11-18
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-11-18
AI Technical Summary
In the edge computing scenario, the computing power of a single edge server is not enough to process video streams in real time, resulting in high data transmission overhead and insufficient computing resources. Especially in the cloud computing mode, the cloud server has a high load pressure and is difficult to meet multiple terminals. Computational requirements for equipment.
The collaborative computing method of multiple edge servers is adopted to realize extensible distributed dual queue dynamic allocation through container technology, and the gateway service is used to schedule video frames to idle servers for deep neural network reasoning, and trigger alarms to expand cluster capacity when computing power is insufficient. Ensure full matching and rapid scaling of computing resources.
It improves the computing resource utilization rate of edge servers, reduces data transmission overhead, realizes more efficient video stream processing capabilities, can meet the computing needs of multiple video streams, and supports rapid capacity expansion and the upgrade of inference services.
Smart Images

Figure CN112463293B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of edge computing and deep learning technology, and in particular to a method for processing a video stream by using a deep learning algorithm in a collaborative manner using an edge server cluster based on container technology in an edge computing scenario. Background Art
[0002] With the development of deep learning theory, deep neural networks have been widely used in many fields, especially in the field of image processing, where they can achieve good results in solving problems such as target recognition and image segmentation. However, deep neural networks usually require huge amounts of computation. When processing video streams continuously, ordinary terminal devices often face the problem of insufficient computing power. Cloud computing provides a way to solve the problem of insufficient computing power. Cloud servers can be equipped with high-performance CPUs and GPUs to provide powerful computing power. However, in the cloud computing model, cloud servers are far away from terminal devices, and using them for video stream processing will result in higher data transmission overhead. Especially when cloud servers are connected to a large number of terminal devices, data transmission will become a bottleneck for the entire processing process.
[0003] Edge computing aims to solve the data transmission overhead caused by the terminal distance in cloud computing, and proposes to set up edge servers close to terminal devices to provide powerful computing support for terminal devices. The benefits of edge computing are that the edge servers are close to the terminals, which shortens the data transmission distance and reduces the transmission overhead; on the other hand, edge computing relieves the load pressure of cloud servers by setting up multiple edge servers within the service range of a cloud server.
[0004] Currently, in the edge computing mode, edge servers are usually used to independently process video streams. Their computing power must exceed the computing power required by the video in order to perform real-time calculations. Summary of the invention
[0005] In order to solve the shortcomings of the prior art, based on the computing mode of edge computing, a collaborative computing method of multiple edge servers is provided for the task of video stream processing using a deep learning algorithm, so as to improve the utilization efficiency of the computing resources of the edge servers and achieve a more adequate matching of computing tasks and computing resources. The present invention adopts the following technical solutions:
[0006] The container-based scalable distributed dual-queue dynamic allocation method in edge scenarios includes the following steps:
[0007] S1, obtain video stream data;
[0008] S2, the gateway service obtains video frames from the video stream data and selects an idle server in the cluster from the idle server queue to process the video frames;
[0009] S3: The inference service on the idle server performs deep neural network inference and result processing on the video frame, and reports to the gateway service that the current server is idle after completion;
[0010] S4, the gateway service will report the idle servers to rejoin the idle server queue.
[0011] Compared with the traditional computing mode in which a single video stream corresponds to a single server, the collaborative computing mode of servers that form a cluster for unified deep neural network inference makes full use of the remaining computing resources of the server and can integrate the remaining computing resources of multiple servers to complete the processing of more video streams.
[0012] The step S2 comprises the following steps:
[0013] S21, after the inference service of each server in the cluster is started, it will notify the gateway service, and the gateway service will open the listening port in the cluster, receive the idle status information of each server in the cluster, and cache it in the idle server queue;
[0014] S22, the gateway service obtains the configuration of multiple terminal devices, records the frame rate of each terminal device, and periodically obtains video frames from the terminal device according to the frame rate and caches them into the input video frame queue; the video frames are read according to the frame rate because the frame rate records the number of frames per second of the terminal device video stream, that is, how many frames are refreshed per second. If the acquisition is not performed according to the frame rate, the acquisition frequency will be too high, which will lead to repeated calculations, and the acquisition frequency will be too low, which will lead to missed inferences. The periodic reading is because the frame rates of different terminal devices are not the same. At the same time, in addition to being related to the frame rate of the terminal device, the gateway service will also define whether to skip frames and how many frames to skip.
[0015] S23, when the gateway service finds that the input video frame queue and the idle server queue are not empty at the same time, it takes out the video frame from the input video frame queue, takes out the idle state information from the idle server queue, and sends the taken out video frame to the idle server corresponding to the taken out idle state information. The inference service in the idle server performs calculation and deletes the video frame taken out from the input video frame queue and the idle state information taken out from the idle server queue.
[0016] The process also includes step S5, where the gateway service periodically checks the length of the input video frame queue of the acquired video frames to determine whether the length of the input video frame queue continues to grow. If so, an alarm is triggered. By checking whether the length of the input video frame queue continues to grow, it is determined whether the computing power of the servers in the cluster meets the computing needs. If it continues to grow, it indicates that the computing power is insufficient, and an alarm is triggered to notify the operation and maintenance personnel to expand the cluster.
[0017] After the alarm is triggered, the cluster is expanded through containerization to support rapid expansion of the cluster, including the following steps:
[0018] S511, installing and starting the container service component on the server system;
[0019] S512, load the inference service image and start the inference service.
[0020] When you need to upgrade the inference service of the edge cluster, you can also upgrade the cluster in a containerized manner, which makes it easier to upgrade the inference service. The steps include:
[0021] S521, downloading the image of the new inference service to the server in the cluster;
[0022] S522, restart the inference service on each server.
[0023] In the step S1, multiple terminal devices perform data preprocessing to convert the read video into video stream data, and the gateway service reads the video frames to be processed by multiple terminal devices.
[0024] The advantages and beneficial effects of the present invention are:
[0025] Multiple adjacent edge servers that can communicate with each other at high speed in the same network environment are formed into an edge server cluster to jointly process the video computing tasks of terminal devices, making full use of the computing power of the edge server cluster to further improve computing efficiency. Specifically, it includes: using the characteristics of edge servers being geographically adjacent and able to access each other at high speed to form an edge server cluster, and globally deploying computing resources to fully meet the needs of computing tasks; triggering an alarm for insufficient computing power, and notifying the operation and maintenance personnel in time to expand the edge server cluster when computing power is insufficient; using container technology to deploy edge inference services and gateway services to meet the needs of rapid expansion and upgrades. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 It is an overall architecture diagram of the edge cluster and terminal camera devices in the present invention.
[0027] Figure 2 It is a timing diagram of the reasoning tasks and result processing tasks of each edge server when no cluster is formed in the present invention.
[0028] Figure 3 It is a timing diagram of the reasoning tasks and result processing tasks of each edge server in the edge cluster after the cluster is formed in the present invention. DETAILED DESCRIPTION
[0029] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.
[0030] like Figure 1 As shown, a scalable distributed dual-queue dynamic allocation method based on containers in edge scenarios uses edge-to-edge collaboration to perform distributed reasoning on multiple video streams. Deep learning is used to perform video stream detection tasks, including three stages: data preprocessing, deep neural network reasoning, and result processing. The end-side device performs data preprocessing on the video and transcodes the pictures collected in the camera into a video stream. After the video stream is received on the edge, the allocation of deep neural network reasoning and result processing tasks within the cluster is determined according to the status of the computing resources of each node in the edge server cluster. The present invention builds an edge cluster on the edge side, realizes flexible and controllable dynamic allocation of computing resources, and improves the efficiency of resource utilization. The implementation method is simple, the means are flexible, and it is independent of the actual running reasoning model. At the same time, the reasoning service runs in the edge cluster in the form of a docker container. This solution has good scalability and the reasoning service is easy to upgrade. The following steps are included:
[0031] Step 1: Multiple terminal camera devices perform data preprocessing, read videos in sequence, and convert the videos into RSTP video stream data;
[0032] Step 2: The gateway service obtains the address of the RSTP video stream according to the configuration, reads the video frames to be processed from multiple terminal camera devices in turn, selects an idle server according to the status of each server in the current cluster, and informs the inference service running on it to process the video frame of the current terminal camera device;
[0033] Step 3: The inference service on the server performs deep neural network inference and result processing on the assigned video frame. After completion, it informs the gateway service that the current server is idle and waits for the next frame of video data to be assigned.
[0034] Step 4: After receiving the notification that the server is in an idle and available state, the gateway service adds the server to the idle server queue again.
[0035] Step 5: The gateway service periodically determines whether the computing power in the cluster meets the computing needs, that is, whether it meets the requirements for processing the corresponding multiple video streams. If not, an alarm is triggered to notify the operation and maintenance personnel to expand the edge server cluster. Specifically, the gateway service periodically checks the length of the input video frame queue, determines whether the length of the input video frame queue continues to grow, and then sends an alarm to the configured alarm source to notify the operation and maintenance personnel to expand the edge cluster; for example, the gateway service records the length of the input video frame queue at a certain time interval, and keeps the most recent 10 values. Each time the queue length is recorded, it also determines whether these 10 values are monotonically increasing. If they are monotonically increasing, it is considered that the input video frame queue is growing continuously.
[0036] Through containerization, operation and maintenance personnel can quickly expand edge clusters and easily upgrade inference services. For example, they can quickly start inference services through containerization and upgrade containers by replacing images.
[0037] Specifically, after receiving the alarm of insufficient computing power, the operation and maintenance personnel prepare a new edge server and place the edge server and the current edge server cluster in the same network environment; install Docker and other components on the new edge server, download the corresponding inference service image, and start the inference service; at the same time, when the inference service needs to be upgraded, the operation and maintenance personnel download the new inference service image to all servers in the cluster, and restart the inference service on each server in turn.
[0038] The process of a new edge server joining the cluster is as follows:
[0039] 1. Install the Linux operating system, such as Ubuntu or CentOS;
[0040] 2. Use apt or yum to install the docker service, and use systemctl to set the docker service to start automatically at boot;
[0041] 3. Use the image package to import the node image;
[0042] 4. Use the docker command to start the container of the corresponding node image and set the container to start automatically.
[0043] The role of the working container on the node:
[0044] 1. Actively register with the gateway service and maintain heartbeat;
[0045] 2. Receive the image sent by the gateway service and perform reasoning on it;
[0046] 3. After the inference is completed and the results are output, the gateway service is notified that the current node is in an idle state.
[0047] Step 2 is the core of the present invention and is divided into the following sub-steps:
[0048] Step 1: After the inference service of each node in the edge server cluster is started, it will inform the gateway service. The gateway service will open the listening port in the cluster, receive the idle status information of each server in the cluster, and cache it in the idle server queue in the memory.
[0049] Step 2: The gateway service reads the configuration of multiple terminal camera devices and records the frame rate FPS of each terminal camera. According to the frame rate FPS, the to-be-processed video frames are periodically read from the corresponding camera and cached into the input video frame queue in the memory. The reason why the video frames are read according to the frame rate FPS is that the frame rate FPS records the number of frames per second of the camera video stream, that is, the camera will refresh several frames per second. If the frame rate FPS is not obtained, the acquisition frequency will be too high, which will lead to repeated calculations, and the acquisition frequency will be too low, which will lead to missed inferences. The periodic reading is because the frame rate FPS of different cameras is different. At the same time, in addition to being related to the frame rate FPS of the camera, the gateway service will also define whether to skip frames and how many frames to skip.
[0050] For example, if the camera is a common camera and the FPS is 30 frames, and the frame skipping setting is not enabled, the gateway service reads the image every 1 / 30 second.
[0051] If the FPS is 30 frames and frame skipping is enabled, assuming that 1 frame is skipped for every 3 frames read, the gateway service first obtains 3 frames, then rests for 1 frame, continues to obtain 3 frames, and then rests for 1 frame. The time intervals for regularly reading images are 1 / 30s, 1 / 30s, 2 / 30s, 1 / 30s, 1 / 30s, 2 / 30s... in this cycle, with 4 / 30s as a repeat period.
[0052] In step 3, when the gateway service finds that the input video frame queue and the idle server queue are not empty at the same time, it takes out the first video frame from the input video frame queue, takes out the information of the first idle server from the idle server queue, sends the taken video frame to the inference service of the idle server for calculation, and deletes the first video frame in the video frame queue and the first idle server in the idle server queue.
[0053] The video stream detection task is performed using deep learning, including three stages: data preprocessing, deep neural network reasoning, and result processing. The end-side device performs data preprocessing on the video and transcodes the pictures collected in the camera into a video stream. After receiving the video stream on the edge side, the allocation of deep neural network reasoning and result processing tasks within the cluster is determined according to the state of the computing resources of each node in the edge server cluster. The present invention builds an edge cluster on the edge side, realizes flexible and controllable dynamic allocation of computing resources, improves the efficiency of resource utilization, and its implementation method is simple, the means are flexible, and it is independent of the actual running reasoning model. At the same time, the reasoning service runs in the edge cluster in the form of a container. This solution has good scalability and the reasoning service is easy to upgrade.
[0054] Compared with the traditional computing mode in which a single video stream corresponds to a single edge server, the collaborative computing mode of edge servers that form a cluster for unified deep neural network inference makes full use of the remaining computing resources of the edge servers. It can integrate the remaining computing resources of multiple edge servers to complete the processing of more video streams.
[0055] Specifically, in the collaborative computing mode of the edge cluster, the yolov3-spp model in the deep neural network is used to process the video stream target recognition task; this embodiment uses Nvidia Jetson AGX Xavier as the edge server, and an 8-megapixel 4K camera as the terminal camera device. The input video stream size is 2080×1920×3, and the video frame rate is 25FPS.
[0056] Without forming a cluster, the edge server takes 65ms to process one frame of video, and can process about 15 frames per second. When doing target recognition, assuming that the target moves slowly, frame skipping can be performed. After frame skipping, one frame is skipped for each frame read, and the required processing speed is 12.5 frames per second. At this time, the edge server meets the requirements of deep neural network reasoning and result processing tasks for a single video stream.
[0057] At this time, five edge servers are formed into an edge cluster. Since the gateway service does not use the GPU function, the gateway service can be deployed on any server without affecting the efficiency of the reasoning task. The reasoning service is deployed on all servers, and the gateway service dynamically allocates the computing tasks. Figure 2 , 3 As shown in the figure, after processing several frames, the reasoning task of the edge service cluster will tend to a steady state, and there will be no idle time gaps in the edge server. At this time, the capabilities of the edge server can be fully utilized. Five edge servers can complete the computing task of 75 frames per second and calculate the video streams of a total of 6 terminal camera devices.
[0058] The analysis results of the test data in this embodiment show that:
[0059] Compared with non-clustered video streams, the resources of the clustered edge servers are used more fully, with the original computing speed of 12.5 frames per second per edge server increased to the maximum value of 15 frames per second for edge servers, making full use of the computing resources of the edge servers and improving performance by 20%.
[0060] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A scalable distributed dual-queue dynamic allocation method based on containers in edge scenarios, characterized by The steps include: S1, obtain video stream data; S2, the gateway service obtains video frames from the video stream data and selects an idle server in the cluster from the idle server queue to process the video frames, including the following steps: S21, after the inference service of each server in the cluster is started, it will notify the gateway service, and the gateway service will open the listening port in the cluster, receive the idle status information of each server in the cluster, and cache it in the idle server queue; S22, the gateway service obtains the configuration of multiple terminal devices, records the frame rate of each terminal device, and periodically obtains video frames from the terminal devices according to the frame rate and caches them into the input video frame queue; S23, when the gateway service finds that the input video frame queue and the idle server queue are not empty at the same time, it takes out the video frame from the input video frame queue, takes out the idle server status information from the idle server queue, sends the taken out video frame to the idle server corresponding to the taken out idle server status information, and the inference service in the idle server performs calculation, and deletes the video frame taken out from the input video frame queue and the idle server status information taken out from the idle server queue; S3: The inference service on the idle server performs deep neural network inference and result processing on the video frame. After completion, it reports to the gateway service that the current server is idle. S4, the gateway service will report the idle servers to rejoin the idle server queue.
2. The scalable distributed dual-queue dynamic allocation method based on containers in edge scenarios as described in claim 1 is characterized in that The method further includes step S5, wherein the gateway service periodically checks the length of the input video frame queue of the acquired video frames to determine whether the length of the input video frame queue continues to grow, and if so, triggers an alarm.
3. The scalable distributed dual-queue dynamic allocation method based on containers in edge scenarios as described in claim 2 is characterized in that After the alarm is triggered, the cluster is expanded through containerization, including the following steps: S511, installing and starting the container service component on the server system; S512, load the inference service image and start the inference service.
4. The scalable distributed dual-queue dynamic allocation method based on containers in edge scenarios as described in claim 2 is characterized in that When you need to upgrade the inference service, upgrade the cluster through containerization, including the following steps: S521, downloading the image of the new inference service to the server in the cluster; S522, restart the inference service on each server.
5. The scalable distributed dual-queue dynamic allocation method based on containers in edge scenarios as described in claim 1 is characterized in that In the step S1, multiple terminal devices perform data preprocessing to convert the read video into video stream data, and the gateway service reads the video frames to be processed by multiple terminal devices.
