A service recovery method and apparatus

Abstract

This invention provides a service recovery method and apparatus, relating to the field of video processing technology. The method is applied to a gateway device and includes: in response to a successful restart of the gateway device, loading task metadata of unfinished video processing tasks stored in a data file into the memory of the gateway device; recreating a target task for processing a target video stream acquired by a target video acquisition device based on the loaded task metadata; receiving the target video stream sent by a server, and processing the received target video stream according to the recreated target task, wherein the server stores the target video stream uploaded by the target video acquisition device. By applying the service recovery scheme provided in this application, the gateway device can restore service after a restart.

Description

Technical Field

[0001] This invention relates to the field of video processing technology, and in particular to a service restoration method and apparatus. Background Technology

[0002] In scenarios such as remote video inspection, emergency command, and security monitoring, video acquisition devices capture video streams in real time and push them to a backend server, allowing users to view the video stream through the server. In addition to the video acquisition devices and the backend server, the aforementioned communication architecture can also include a gateway device. This gateway device provides users with services such as video storage, video analysis, and video forwarding.

[0003] The gateway device communicates with the backend server. After receiving a new video stream, if the backend server needs to store, analyze, forward, or perform other processing on the video stream, it will instruct the gateway device to create a video processing task. After the task is created, the gateway device will distribute the video stream to the gateway device for processing.

[0004] However, if the gateway device restarts due to malfunction, version update, or other reasons, the video processing tasks it created will be lost. Furthermore, the backend server is only responsible for transmitting video streams to the gateway device and does not detect whether the video processing tasks on the gateway device are lost. Therefore, the backend server will not instruct the gateway device to recreate the video processing tasks. As a result, the gateway device cannot continue processing video streams after restarting, leading to service interruption. Therefore, a service recovery scheme is needed to restore service after the gateway device restarts. Summary of the Invention

[0005] The purpose of this invention is to provide a service recovery method and apparatus to restore services after a gateway device restarts. The specific technical solution is as follows:

[0006] In a first aspect, embodiments of this application provide a service recovery method applied to a gateway device, the method comprising:

[0007] In response to the successful restart of the gateway device, the task metadata of the unfinished video processing task stored in the data file is loaded into the memory of the gateway device. The data file is persistently stored on the local disk. The task metadata of the unfinished video processing task includes data related to the target video acquisition device that acquires the video stream processed by the unfinished video processing task.

[0008] Based on the loaded task metadata, a new target task is created for processing the target video stream acquired by the target video acquisition device.

[0009] The server receives the target video stream sent by the server and processes the received target video stream according to the recreated target task. The server stores the target video stream uploaded by the target video acquisition device.

[0010] In one embodiment of this application, before the gateway device restarts, the following is further included:

[0011] The server receives a task creation instruction after it first receives the target video stream, wherein the task creation instruction carries data related to the target video acquisition device;

[0012] The target task is created based on the data carried by the task creation instruction;

[0013] Generate the task metadata for the target task;

[0014] The task metadata of the target task is stored in the data file, and the server is notified that the task creation is complete;

[0015] Receive the target video stream pushed by the server, and process the target video stream according to the target task.

[0016] In one embodiment of this application, the task creation instruction further carries a target type for processing the target video stream;

[0017] The step of creating the target task based on the data carried by the task creation instruction includes:

[0018] Based on the data carried by the task creation instruction, create a target task that processes the target video stream according to the target type;

[0019] The task metadata for generating the target task includes:

[0020] Generate task metadata containing the target type;

[0021] The processing of the target video stream according to the target task includes:

[0022] The target video stream is processed according to the processing method corresponding to the target type.

[0023] In one embodiment of this application, the method further includes:

[0024] Receive the notification from the server that the target video acquisition device has finished acquiring the video stream;

[0025] Clear the target task and delete the task metadata of the target task stored in the data file.

[0026] In one embodiment of this application, the task metadata of the target task includes the following data:

[0027] The device identifier of the target video acquisition device or the task identifier of the task performed by the target video acquisition device;

[0028] The video stream identifier of the target video stream;

[0029] The address information of the server.

[0030] Secondly, embodiments of this application provide a service recovery device applied to a gateway device, the device comprising:

[0031] The data loading module is used to load the task metadata of the unfinished video processing task stored in the data file into the memory of the gateway device in response to the successful restart of the gateway device. The data file is persistently stored on the local disk, and the task metadata of the unfinished video processing task includes data related to the target video acquisition device that acquires the video stream processed by the unfinished video processing task.

[0032] The task reconstruction module is used to recreate the target task for processing the target video stream acquired by the target video acquisition device based on the loaded task metadata.

[0033] The first processing module is used to receive the target video stream sent by the server, and process the received target video stream according to the recreated target task. The server stores the target video stream uploaded by the target video acquisition device.

[0034] In one embodiment of this application, the apparatus further includes:

[0035] The instruction acquisition module is used to acquire, before the gateway device restarts, a task creation instruction issued by the server after it first receives the target video stream, wherein the task creation instruction carries data related to the target video acquisition device;

[0036] The task creation module is used to create the target task based on the data carried by the task creation instruction;

[0037] The data generation module is used to generate the task metadata of the target task;

[0038] The data storage module is used to store the task metadata of the target task into the data file and notify the server that the task creation is complete.

[0039] The second processing module is used to receive the target video stream pushed by the server and process the target video stream according to the target task.

[0040] In one embodiment of this application, the task creation instruction further carries a target type for processing the target video stream;

[0041] The task creation module is specifically used for:

[0042] Based on the data carried by the task creation instruction, create a target task that processes the target video stream according to the target type;

[0043] The data generation module is specifically used for:

[0044] Generate task metadata containing the target type;

[0045] The second processing module is specifically used for:

[0046] The target video stream is processed according to the processing method corresponding to the target type.

[0047] In one embodiment of this application, the apparatus further includes:

[0048] The notification receiving module is used to receive the notification from the server that the video stream acquisition of the target video acquisition device has ended;

[0049] The data deletion module is used to clear the target task and delete the task metadata of the target task stored in the data file.

[0050] In one embodiment of this application, the task metadata of the target task includes the following data:

[0051] The device identifier of the target video acquisition device or the task identifier of the task performed by the target video acquisition device;

[0052] The video stream identifier of the target video stream;

[0053] The address information of the server.

[0054] Thirdly, embodiments of this application provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0055] Memory, used to store computer programs;

[0056] When a processor executes a program stored in memory, it implements any of the methods described in the first aspect above.

[0057] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the methods described in the first aspect above.

[0058] Beneficial effects of the embodiments of the present invention:

[0059] As can be seen from the above, when the solution provided in the embodiments of this application is used to restore services, the data file stores the task metadata of the unfinished video processing task. After the gateway device restarts, it can recreate the task according to the task metadata stored in the data file, and then continue to process the video stream according to the recreated task. This achieves the purpose of restoring services after the gateway device restarts, and improves the continuity and reliability of the gateway device in processing video streams and providing services. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0061] Figure 1 A schematic diagram illustrating the communication architecture between a video acquisition device, a server, and a gateway device provided in this application embodiment;

[0062] Figure 2 A flowchart illustrating the first service recovery method provided in this application embodiment;

[0063] Figure 3 This is a schematic diagram of the structure of a gateway device provided in an embodiment of this application;

[0064] Figure 4a A flowchart illustrating a task creation method provided in an embodiment of this application;

[0065] Figure 4b A flowchart illustrating the second service recovery method provided in this application embodiment;

[0066] Figure 4c A flowchart illustrating a task clearing method provided in an embodiment of this application;

[0067] Figure 5 A flowchart illustrating a video stream processing method for a video acquisition device provided in an embodiment of this application;

[0068] Figure 6 This is a schematic diagram of the structure of a service restoration device provided in an embodiment of this application;

[0069] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0070] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of the present invention.

[0071] First, the application scenarios of the service recovery solution provided in the embodiments of this application will be explained.

[0072] See Figure 1 The diagram illustrates the communication architecture between video capture device 101, server 102, and gateway device 103. Server 102 can be a main service, such as a WebRTC server; gateway device 103 can be any media processing gateway used for processing video streams, such as storage, analysis, and forwarding.

[0073] In the above communication architecture, the video acquisition device 101 is communicatively connected to the server 102, and the link between them can be called the main link. The video acquisition device 101 is responsible for performing video acquisition tasks, such as video inspection, emergency command, and security monitoring. During the execution of tasks, the video acquisition device 101 acquires video streams in real time and pushes the video streams to the server 102 in real time. The server 102 can provide a bypass to communicate with the gateway device 103. After receiving the video stream, the server 102 can display the video stream in real time for users to view through the backend server, and also distribute the video stream to the gateway device 103 through the bypass. After receiving the video stream, the gateway device 103 can process the video stream accordingly.

[0074] When the video acquisition device 101 is a drone, after receiving the video stream, the server 102 displays the video stream to allow the drone operator to observe from a first-person perspective with low latency and control the drone. On the other hand, it can distribute the video stream to the gateway device 103 through a bypass. After receiving the video stream, the gateway device 103 can process the video stream accordingly.

[0075] In addition, the video acquisition device 101 mentioned above can also be other devices used for video acquisition besides drones, and this application embodiment does not limit this.

[0076] The following three scenarios may occur in the above communication architecture:

[0077] In the first scenario, server 102 is communicatively connected to multiple video capture devices 101, receiving video streams pushed by the devices. Server 102 can also push multiple video streams to gateway device 103. In this scenario, gateway device 103 receives the multiple video streams pushed by server 102 and processes them.

[0078] In the second scenario, multiple servers 102 can communicate with the same gateway device 103. In this case, the gateway device 103 receives multiple video streams pushed by multiple servers 102 and processes the received video streams.

[0079] In the third scenario, the same server 102 communicates with multiple gateway devices 103, allowing the server 102 to distribute the same video stream to these devices. In this case, each gateway device 103 essentially receives the video stream pushed by the server 102 and processes it.

[0080] When the gateway device restarts, video processing tasks within the gateway device will be lost, causing processes such as video storage and recording, video analysis, and video forwarding to cease, resulting in the loss of critical and important data.

[0081] Although existing technologies have service recovery mechanisms that rely on client reconnection, and in the above communication architecture, the video capture device can be regarded as a client, the streaming link of the video capture device is established with the server. It can be aware of the existence of the server in the communication architecture, but it cannot be aware that the server is also connected to the gateway device in a bypass. Therefore, it is difficult to apply the existing service recovery mechanism to restore the service when the gateway device restarts.

[0082] Therefore, in the above application scenarios, once the gateway device restarts, the services it provides will be permanently interrupted unless manual intervention is carried out, that is, a person intervenes to rebuild the video processing tasks in the gateway device after learning that the gateway device has restarted. However, this is unacceptable for services such as automated inspection of video acquisition equipment that require 24 / 7 high availability.

[0083] To enable gateway devices to resume services after a reboot, this application provides a service recovery method and apparatus, which will be described in detail below through specific embodiments.

[0084] See Figure 2 A flowchart of a service recovery method is provided. In this embodiment, the above method is applied to a gateway device and includes the following steps S201-S203.

[0085] Step S201: In response to the successful restart of the gateway device, load the task metadata of the unfinished video processing tasks stored in the data file into the memory of the gateway device.

[0086] The aforementioned data file is used to store the task metadata of incomplete video processing tasks. For example, the data file could be an rtp_processors.json file. For each incomplete video processing task, as the gateway device continuously executes the task, once the task is completed, the gateway device either deletes the task's metadata from the data file or marks the task's metadata in the data file to indicate that the task has been completed.

[0087] In addition, the aforementioned data files are also persistently stored on the local disk, rather than in the gateway device's memory. Both the local disk and the gateway device's memory are storage spaces within the gateway device. The gateway device's memory can store video streams and data related to video processing tasks. When the gateway device restarts due to malfunctions, version updates, or other reasons, the data stored in the gateway device's memory will be cleared. After restarting, as the gateway device recreates video processing tasks and receives video streams, the gateway device's memory will again store the video streams and data related to the newly created video processing tasks. The local disk within the gateway device can persistently store data, ensuring that the stored data will not be lost even if the gateway device restarts. Therefore, persistently storing the aforementioned data files on the local disk guarantees that the data files will not be lost.

[0088] The task metadata for the aforementioned incomplete video processing tasks includes data related to the target video acquisition device that acquired the video stream processed by the incomplete video processing task. A description of the task metadata can be found in subsequent embodiments, and will not be detailed here.

[0089] Specifically, before the gateway device restarts, the gateway device's memory stores all the unfinished video processing tasks, and the data files saved on the local disk store the task metadata of each unfinished video processing task. When the gateway device restarts, there are no video processing tasks in the gateway device's memory, but the data files saved on the local disk still store the task metadata of each unfinished video processing task. At this time, the gateway device can respond to its successful restart by reading the data files and loading the task metadata of the unfinished video processing tasks stored in the data files into the gateway device's memory.

[0090] Step S202: Based on the loaded task metadata, recreate the target task for processing the target video stream acquired by the target video acquisition device.

[0091] Specifically, the gateway device can execute multiple video processing tasks simultaneously. Therefore, the data file can store the task metadata of multiple incomplete video processing tasks. After the gateway device loads the task metadata stored in the data file into memory, for each incomplete video processing task, the gateway device can recreate a target task for processing the target video stream captured by the target video capture device based on the task metadata of that incomplete video processing task. The recreated target task is actually the same as the incomplete video processing task, and it processes the video stream (i.e., the target video stream) captured by the same video capture device (i.e., the target video capture device).

[0092] For example, the data file stores the task metadata for two incomplete data processing tasks, P1 and P2. Task P1 processes the video stream captured by video acquisition device Q1, and task P2 processes the video stream captured by video acquisition device Q2. Based on the task metadata of task P1, the gateway device can recreate the target task for processing the video stream captured by video acquisition device Q1; and based on the task metadata of task P2, it can recreate the target task for processing the video stream captured by video acquisition device Q2.

[0093] Step S203: Receive the target video stream sent by the server and process the received target video stream according to the recreated target task.

[0094] The server stores the target video stream uploaded by the target video acquisition device.

[0095] From the above-mentioned targets Figure 1 As can be seen from the description, the video capture device pushes the video stream to the server, and the server can store the video stream pushed by the video capture device for display.

[0096] Specifically, after the gateway device restarts, it first creates a target task. After the target task is created, it starts receiving the target video stream sent by the server. This ensures that the processing environment in the gateway device's memory is ready before receiving the video stream. As a result, when receiving the video stream, it can immediately process the received video stream according to the newly created task, ensuring that critical data is not lost and achieving seamless recovery.

[0097] As can be seen from the above, when using the solution provided in this application embodiment for service restoration, the data file stores the task metadata of the unfinished video processing task. After the gateway device restarts, it can recreate the task based on the task metadata stored in the data file, and then continue processing the video stream based on the recreated task. This achieves the goal of restoring service after the gateway device restarts, improving the continuity and reliability of the gateway device in processing video streams and providing services. Furthermore, the entire restoration process is completed independently by the gateway device. Therefore, this solution can be considered a self-recovery solution for gateway device services. Especially in various automated business scenarios of video acquisition devices (such as 24 / 7 unattended automated tasks), the self-healing capability of the service restoration solution provided in this application embodiment is key to ensuring business continuity and high reliability. In various application scenarios of video acquisition devices, the service restoration solution provided in this application embodiment can autonomously restore services after the gateway device restarts, effectively avoiding the interruption of video recording and data loss for tasks such as inspection and surveying of video acquisition devices caused by gateway device restarts. This ensures the continuity of recording and analysis tasks, and thus guarantees the integrity of critical video data. This solution has significant application value.

[0098] In scenarios where the video acquisition device is a drone, compared to existing service recovery mechanisms that rely on client reconnection, the service recovery solution provided in this application addresses the recovery challenges caused by the lack of direct signaling between the gateway device and the drone in drone scenarios. It achieves service recovery that is completely transparent to the video stream sender (drone and server) and is performed solely on the gateway device side. Furthermore, because this solution is completed on the gateway device side, it is completely transparent to both the drone and server sides and has no impact on them. Therefore, no modifications are required to the drone flight control, image transmission, or main link (the link between the drone and the server), significantly reducing the cost and complexity of integrating and implementing this solution in existing drone video application scenarios.

[0099] The following section uses the task metadata of the target task as an example to introduce the task metadata used in the above method.

[0100] First, the target task, target video stream, and target video acquisition device will be explained.

[0101] As can be seen from the descriptions of the first and second application scenarios above, the gateway device can receive multiple video streams, that is, receive video streams collected by multiple video capture devices. These video streams are distributed to the gateway device by one or more servers. The gateway device needs to process the video streams collected by the multiple video capture devices. Each video stream that needs to be processed is a target video stream, and the video capture device corresponding to each target video stream (i.e., the video capture device that captures the target video stream) is a target video capture device. Each target task is used to process the video stream captured by one target video capture device.

[0102] For example, if the gateway device's memory contains three video processing tasks that are currently being executed (i.e., incomplete video processing tasks) to process video streams captured by three video capture devices, then these three video processing tasks are the three target tasks, and these three video capture devices are the three target video capture devices.

[0103] In one embodiment of this application, the task metadata of the target task includes the following three types of data:

[0104] The first type of data: the device identifier of the target video acquisition device or the task identifier of the task performed by the target video acquisition device. For example, if the video acquisition device is a drone, the task identifier could be RoomSid, RoomId, etc.

[0105] When the task metadata of the target task includes the device identifier of the target video acquisition device, a target task is created for each target video acquisition device based on that device identifier. This allows the created target task to focus on processing the video stream acquired by that target video acquisition device. After each target task is created, each target task focuses on processing the video stream acquired by its corresponding target video acquisition device, thereby ensuring that the gateway device processes multiple video streams in an orderly and stable manner.

[0106] When the task metadata of the target task contains the task identifier of the task executed by the target video acquisition device, since there is a one-to-one correspondence between the target video acquisition device and the task it executes, creating the target task based on the task identifier also enables each target task to process the video stream acquired by its corresponding target video acquisition device.

[0107] The second type of data: the video stream identifier of the target video stream. For example, the Ssrc (Synchronization Source) media stream identifier.

[0108] The video stream identifier of the target video stream can be considered as a unique identifier for the target video stream. For each target video stream, a target task is created based on the video stream identifier of that target video stream. This allows the created target task to focus on processing that target video stream. After all target tasks are created, each target task can focus on processing its corresponding target video stream, thereby ensuring that the gateway device processes multiple video streams in an orderly and stable manner.

[0109] The third type of data: server address information.

[0110] The address information mentioned above may include the server's source IP address and the server's push port information, etc.

[0111] A single gateway device can connect to multiple servers. Furthermore, other devices connected to the gateway device, besides the servers, can also push video streams to it. Therefore, when creating a target task, it can be created based on the server's address information. This allows the gateway device to filter the received video streams based on the server's address information. This not only enables the gateway device to process video streams quickly but also filters out maliciously transmitted data by attackers, preventing the gateway device from being attacked and malfunctioning, thus enhancing the gateway device's security.

[0112] Before restarting the gateway device, you can also perform the steps in the solution provided in the following embodiments.

[0113] In one embodiment of this application, before the gateway device restarts, a task creation instruction issued by the server after the target video stream is first received can be obtained; a target task is created according to the data carried by the task creation instruction; task metadata of the target task is generated; the task metadata of the target task is stored in a data file, and the server is notified that the task creation is complete; the target video stream pushed by the server is received, and the target video stream is processed according to the target task.

[0114] The task creation instruction contains data related to the target video acquisition device.

[0115] Specifically, when the server receives the target video stream for the first time, if it needs to perform processing such as recording, analysis, or forwarding on the target video stream, the server can generate a task creation instruction based on the data related to the target video acquisition device that acquired the target video stream, and send the task creation instruction to the gateway device.

[0116] In one implementation, after receiving an instruction from a user to process the target video stream, the server can generate a task creation instruction and send the task creation instruction to the gateway device.

[0117] In another implementation, the server can pre-obtain information indicating whether video streams captured by each video capture device need to be processed. When the server first receives a video stream from a particular video capture device, it can determine, based on the pre-obtained information, whether processing is necessary for that specific video stream. If the server determines that processing is required, it generates a task creation instruction and sends it to the gateway device.

[0118] For example, if the server has already determined that it needs to process the video stream captured by video capture device A, but does not need to process the video stream captured by video capture device B, then when the server first receives the video stream captured by video capture device A, it can quickly determine that the received video stream needs to be processed, thereby generating a task creation instruction and sending the task creation instruction to the gateway device.

[0119] After receiving the task creation instruction, the gateway device can create a target task based on the data related to the target video acquisition device carried in the task creation instruction, generate the target task's task metadata, such as the device identifier, video stream identifier, address information, etc. of the target video acquisition device mentioned in the previous embodiment, store the generated task metadata in a data file, and notify the server that the task creation is complete.

[0120] Upon receiving the notification, the server begins pushing the target video stream to the gateway device. The gateway device receives the target video stream pushed by the server and processes it according to the created target task.

[0121] As can be seen from the above, when the solution provided in the embodiments of this application is used for service restoration, after the server receives the target video stream for the first time, it instructs the gateway device to create a target task. It can be seen that the target task is also the first video processing task created by the gateway device to process the target video stream. After creating the target task, the gateway device stores the task metadata of the target task in a data file. In this way, even if the gateway device restarts, it can reconstruct the target task according to the task metadata stored in the data file and continue to process the target video stream, thereby realizing the restoration of service after the gateway device restarts.

[0122] In one embodiment of this application, the task creation instruction also carries a target type for processing the target video stream.

[0123] The aforementioned target types can be video processing types that process video streams, such as video storage, analysis, and RTMP (Real-Time Messaging Protocol) distribution. In a communication architecture where the server is a WebRTC server, the aforementioned video processing type can be OutputType.

[0124] Similar to the server determining whether the target video stream needs to be processed as mentioned in the previous embodiments, the server can also determine what kind of processing is needed for the target video stream, that is, determine the target type, and thereby generate a task creation instruction carrying information related to the target video acquisition device and the target type.

[0125] In one implementation, the server can obtain the video processing type specified by the user as the target type. This method involves the user specifying the target type for processing the target video stream.

[0126] In another implementation, the server can pre-store the correspondence between each video capture device and the video processing type. Then, after the server first receives the target video stream captured by the target video capture device, it can determine the target type based on this correspondence.

[0127] For example, the server pre-stores a mapping relationship where video capture device A corresponds to video recording type, and video capture device B corresponds to video analysis type. Once the server receives the target video stream captured by video capture device B for the first time, it can determine that the target type is video analysis type.

[0128] When the task creation instruction also carries the target type, the gateway device can call the relevant plugins to create a target task that processes the target video stream according to the target type, based on the data carried in the task creation instruction (i.e., information related to the target video acquisition device and the target type).

[0129] For example, if the target type is video analysis, the video analysis plugin is invoked to create a video analysis task; if the target type is video recording, the video recording plugin is invoked to create a video recording task.

[0130] When generating task metadata, in addition to the various identifiers and address information mentioned above, the target type can also be stored as task metadata in the data file.

[0131] Similarly, when processing the target video stream according to the target task, the target video stream can be processed according to the processing method corresponding to the target type.

[0132] For example, if the target type is video analysis, then video analysis is performed on the target video stream; if the target type is video recording, then video recording is performed on the target video stream.

[0133] As can be seen from the above, when using the solution provided in the embodiments of this application to restore services, the task creation instruction issued by the server can also carry the target type, so that the gateway device can create the corresponding task according to the server's instructions, thereby improving the flexibility of processing video streams.

[0134] In one embodiment of this application, the method further includes: receiving a notification from the server that the target video acquisition device has finished acquiring the video stream; clearing the target task and deleting the target task's metadata stored in the data file.

[0135] Specifically, when the target video acquisition device finishes acquiring the video stream, or in other words, when the video acquisition device completes its task, the target video acquisition device stops pushing the video stream to the server. Alternatively, the video acquisition device can notify the server that the video stream acquisition has ended (or the task has been completed). This allows the server to determine that the target video acquisition device has finished acquiring the video stream and then notify the gateway device that the target video acquisition device has finished acquiring the video stream. After receiving the notification from the server, the gateway device can clear the target task from its memory and delete the target task's metadata stored in the data file.

[0136] As can be seen from the above, when performing service restoration using the solution provided in this application embodiment, the gateway device can clear the target task and delete the target task's task metadata stored in the data file after the target video acquisition device finishes acquiring the video stream. This completes the processing of the video stream acquired by the target video acquisition device, and deleting the target task's task metadata stored in the data file ensures that the task metadata stored in the data file is always the task metadata of the incomplete video processing task, avoiding the creation of completed target tasks when the gateway device restarts. Therefore, the service restoration solution provided in this application embodiment can not only restore services when the gateway device restarts, but also ensure the normal execution of the video processing task throughout its entire lifecycle, enabling the gateway device to process data accurately and reliably.

[0137] The complete processing flow of a single video stream is described below. For ease of description, the target video acquisition device will be referred to as video acquisition device M, the target video stream as video stream N, and the target task as task K.

[0138] When video capture device M begins its task, it pushes the captured video stream N to the server. The server receives video stream N for the first time, and therefore sends a task creation command to the gateway device. Upon receiving the command, the gateway device creates task K. After creation, the gateway device stores the task metadata of task K in a data file and notifies the server that task creation is complete. This completes the preparation phase for processing video stream N.

[0139] After the server confirms that the gateway device task has been created, it pushes video stream N to the gateway device. After receiving video stream N, the gateway device processes video stream N according to task K.

[0140] Video capture device M continuously captures and pushes video stream N, while the server continuously pushes video stream N to the gateway device, which in turn continuously processes video stream N. During this period, if the gateway device malfunctions or undergoes a version update, it will be unable to process video stream N until it restarts. The gateway device then loads the task metadata of task K stored in the data file into memory, recreates task K based on the loaded metadata, and after task reconstruction, it can continue to receive video stream N and process it according to task K.

[0141] Once video capture device M completes its task, the server can determine that video capture of video stream N by device M has ended. At this point, the server can notify the gateway device that video capture of video stream N by device M has ended. Upon receiving this notification, the gateway device can clear task K from memory and delete the task metadata of task K stored in the data file.

[0142] A gateway device may contain multiple functional modules, and each step in the service recovery scheme provided in this application embodiment can be executed by these multiple functional modules.

[0143] In one embodiment of this application, see [link to embodiment]. Figure 3 The internal structure of the gateway device is shown. In this embodiment, the gateway device 103 includes a video stream receiving unit 1031, a task manager 1032, and a data persistence module 1033.

[0144] Among them, the video stream receiving unit 1031 is responsible for receiving real-time video streams collected by video acquisition devices from upstream (i.e., the server), such as RTP (Real-time Transport Protocol) packets and RTCP (RTP Control Protocol) packets.

[0145] The task manager 1032 runs in the memory of the gateway device and is responsible for creating, managing and destroying the corresponding task instance (RtpProcessor) for each video stream received by the video stream receiving unit 1031.

[0146] The data persistence module 1033 is responsible for synchronizing information between the task manager 1032 and the persistent state ledger (i.e., data file) on the local disk. This module provides interfaces for adding tasks, deleting tasks, and loading all tasks.

[0147] Figure 3 The document also showcases a local disk 1034, which contains a persistent state ledger used to persistently record the task metadata of all currently unfinished video processing tasks.

[0148] The following is combined Figures 4a-4c This document describes the specific process for implementing service recovery schemes by the various functional modules contained in the gateway device.

[0149] Figure 4a This illustrates the process of creating tasks when the gateway device is running normally. Figure 4a In the process, when the server first receives the video stream captured by the video capture device (step S401), it can issue a task creation command to the task manager. After receiving the task creation command, the task manager creates a video processing task (step S402). After creation, on the one hand, the task manager notifies the server that the task creation is complete; on the other hand, the data persistence module obtains the task metadata of the video processing task, calls the interface for adding tasks (step S403), and writes the task metadata of the video processing task into the persistent state ledger on the local disk (step S404).

[0150] Once the server confirms that the gateway device task has been created, it pushes the video stream to the gateway device. The video stream receiving unit receives the video stream, and the task manager processes the video stream according to the video processing task.

[0151] Figure 4b This illustrates the process for restoring services in the event of a gateway device reboot. Figure 4b In the process, the gateway device restarts (step S405). Before the gateway device starts receiving and processing any data packets, the data persistence module immediately calls the interface for loading all tasks (step S406), reads and parses the persistent state ledger (step S407), traverses the task metadata of the video processing tasks recorded in the ledger (step S408), and recreates the video processing task based on the task metadata of each video processing task traversed (step S409). It is then determined whether all data in the ledger has been traversed (step S410). If not, the process returns to the step of traversing the task metadata of the video processing tasks recorded in the ledger; if yes, the video stream receiving unit starts receiving RTP packets, and the gateway device enters the normal RTP packet receiving and processing loop (step S411), achieving seamless service recovery.

[0152] Figure 4c This illustrates the process of the gateway device clearing the task when the video stream ends. Figure 4cIn the process, the server sends a notification to the task manager to end the acquisition (step S412). Upon receiving the notification, the task manager destroys the video processing task in memory (step S413). The data persistence module determines that the video processing task has ended through the task manager and calls the interface to delete the task (step S414) to delete the task metadata of the video processing task stored in the persistent state ledger (step S415).

[0153] In one embodiment of this application, see [link to embodiment]. Figure 5 This illustrates a video stream processing flow of a video acquisition device. For the gateway device 103, both the video acquisition device 101 and the server 102 are devices in the external environment. Furthermore, the gateway device 103 is also communicatively connected to the downstream application 104, which may include RTMP services, storage, AI analysis services, GB28181 services, etc., and is also an application in the external environment.

[0154] exist Figure 5 In this process, video capture device 101 pushes a video stream to server 102, which can be considered the main video stream; server 102 pushes a video stream to video stream receiving unit 1031 in gateway device 103, which can be considered a bypass video stream. Video stream receiving unit 1031 then transmits the received data packets (video streams) to task manager 1032, which processes the data packets according to the video processing tasks in memory. Task manager 1032 also interacts with data persistence module 1033 to synchronize information when creating and destroying tasks. Data persistence module 1033 can read and write to the persistent state ledger on local disk 1034. Furthermore, task manager 1032 also sends the processing results of the data packets to downstream application 104. Testing showed that in the event of a reboot, the application of this solution can quickly restore the video stream processing mechanism within seconds, greatly reducing the impact on downstream applications 104, making it virtually imperceptible to them (such as monitoring screens and AI analysis systems). Furthermore, the bypass processing system built around the gateway device 103 is completely decoupled from the main link. This bypass processing system is independent of the main link, yet it provides high availability assurance for the main services of the video stream from the video acquisition device 101 (such as WebRTC services), improving the overall system architecture's robustness and making it more flexible and resilient.

[0155] The testing process is as follows:

[0156] In the test environment, the video stream was encoded in H.264, with a resolution of 1080P and a frame rate of 25fps. The bypass stream was forwarded to the gateway device via RTP / RTCP packets. Scenarios were simulated including abnormal exit of the gateway process, active restart of the gateway service, restart after a version update, and abnormal interruption of the bypass stream processing module.

[0157] In the test, the restart of the gateway process was used as the starting point for recovery timing, and the recovery timing ended when the system completed reading the rtp_processors.json state ledger, reconstructing historical active sessions, and enabling RtpProcessor to enter a state where it could receive and process RTP packets. After multiple tests, the recovery time for a single bypass video stream was approximately less than 1 second; when 5 bypass video streams were recovered simultaneously, the recovery time was approximately 1 to 3 seconds; and when 10 bypass video streams were recovered simultaneously, the recovery time was approximately 1 to 5 seconds.

[0158] Therefore, the "rapid recovery within seconds" mentioned in this solution means that after the gateway service restarts, the gateway device can automatically rebuild the bypass stream processing session based on the persistent state ledger and recover to a state where it can continue to process the bypass video stream within seconds, without the need for the video capture device to push the stream again, the server to re-establish the signaling, or the manual re-creation of the bypass video processing task.

[0159] Corresponding to the aforementioned service restoration method, this application also provides a service restoration device.

[0160] In one embodiment of this application, see [link to embodiment]. Figure 6 A schematic diagram of a service recovery device is provided. In this embodiment, the device is applied to a gateway device and includes:

[0161] The data loading module 601 is used to load the task metadata of the unfinished video processing task stored in the data file into the memory of the gateway device in response to the successful restart of the gateway device. The data file is persistently stored on the local disk, and the task metadata of the unfinished video processing task includes data related to the target video acquisition device that acquires the video stream processed by the unfinished video processing task.

[0162] The task reconstruction module 602 is used to recreate the target task for processing the target video stream acquired by the target video acquisition device based on the loaded task metadata.

[0163] The first processing module 603 is used to receive the target video stream sent by the server, process the received target video stream according to the recreated target task, and the server stores the target video stream uploaded by the target video acquisition device.

[0164] As can be seen from the above, when the solution provided in the embodiments of this application is used to restore services, the data file stores the task metadata of the unfinished video processing task. After the gateway device restarts, it can recreate the task according to the task metadata stored in the data file, and then continue to process the video stream according to the recreated task. This achieves the purpose of restoring services after the gateway device restarts, and improves the continuity and reliability of the gateway device in processing video streams and providing services.

[0165] In one embodiment of this application, the apparatus further includes:

[0166] The instruction acquisition module is used to acquire, before the gateway device restarts, a task creation instruction issued by the server after it first receives the target video stream, wherein the task creation instruction carries data related to the target video acquisition device;

[0167] The task creation module is used to create the target task based on the data carried by the task creation instruction;

[0168] The data generation module is used to generate the task metadata of the target task;

[0169] The data storage module is used to store the task metadata of the target task into the data file and notify the server that the task creation is complete.

[0170] The second processing module is used to receive the target video stream pushed by the server and process the target video stream according to the target task.

[0171] Both the first processing module and the second processing module are used to receive the target video stream and process the target video stream according to the target task. The difference is that the second processing module is used to process the target video stream according to the target task created by the task creation instruction issued by the server before the gateway device restarts, while the first processing module is used to process the target video stream according to the recreated target task after the gateway device restarts.

[0172] The first processing module and the second processing module mentioned above can be the same processing module, or they can be two different processing modules.

[0173] As can be seen from the above, when the solution provided in the embodiments of this application is used for service restoration, after the server receives the target video stream for the first time, it instructs the gateway device to create a target task. It can be seen that the target task is also the first video processing task created by the gateway device to process the target video stream. After creating the target task, the gateway device stores the task metadata of the target task in a data file. In this way, even if the gateway device restarts, it can reconstruct the target task according to the task metadata stored in the data file and continue to process the target video stream, thereby realizing the restoration of service after the gateway device restarts.

[0174] In one embodiment of this application, the task creation instruction further carries a target type for processing the target video stream;

[0175] The task creation module is specifically used for:

[0176] Based on the data carried by the task creation instruction, create a target task that processes the target video stream according to the target type;

[0177] The data generation module is specifically used for:

[0178] Generate task metadata containing the target type;

[0179] The second processing module is specifically used for:

[0180] The target video stream is processed according to the processing method corresponding to the target type.

[0181] As can be seen from the above, when using the solution provided in the embodiments of this application to restore services, the task creation instruction issued by the server can also carry the target type, so that the gateway device can create the corresponding task according to the server's instructions, thereby improving the flexibility of processing video streams.

[0182] In one embodiment of this application, the apparatus further includes:

[0183] The notification receiving module is used to receive the notification from the server that the video stream acquisition of the target video acquisition device has ended;

[0184] The data deletion module is used to clear the target task and delete the task metadata of the target task stored in the data file.

[0185] As can be seen from the above, when performing service restoration using the solution provided in this application embodiment, the gateway device can clear the target task and delete the target task's task metadata stored in the data file after the target video acquisition device finishes acquiring the video stream. This completes the processing of the video stream acquired by the target video acquisition device, and deleting the target task's task metadata stored in the data file ensures that the task metadata stored in the data file is always the task metadata of the incomplete video processing task, avoiding the creation of completed target tasks when the gateway device restarts. Therefore, the service restoration solution provided in this application embodiment can not only restore services when the gateway device restarts, but also ensure the normal execution of the video processing task throughout its entire lifecycle, enabling the gateway device to process data accurately and reliably.

[0186] In one embodiment of this application, the task metadata of the target task includes the following data:

[0187] The device identifier of the target video acquisition device or the task identifier of the task performed by the target video acquisition device;

[0188] The video stream identifier of the target video stream;

[0189] The address information of the server.

[0190] In this solution, the target task's metadata includes the three types of data mentioned above, thereby ensuring that the gateway device can process multiple video streams in an orderly and stable manner, and enabling the gateway device to process video streams quickly, filter out data maliciously transmitted by attackers, prevent the gateway device from being attacked and malfunctioning, and enhance the security of the gateway device.

[0191] This invention also provides an electronic device, such as... Figure 7 As shown, it includes a processor 701, a communication interface 702, a memory 703, and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.

[0192] Memory 703 is used to store computer programs;

[0193] When the processor 701 executes the program stored in the memory 703, it implements the steps of any of the above-described service recovery methods.

[0194] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0195] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0196] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0197] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0198] In another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the steps of any of the above-described service recovery methods.

[0199] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the service recovery methods described above.

[0200] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0201] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0202] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0203] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A service restoration method, characterized in that, Applied to a gateway device, the method includes: In response to the successful restart of the gateway device, the task metadata of the unfinished video processing task stored in the data file is loaded into the memory of the gateway device. The data file is persistently stored on the local disk. The task metadata of the unfinished video processing task includes data related to the target video acquisition device that acquires the video stream processed by the unfinished video processing task. Based on the loaded task metadata, a new target task is created for processing the target video stream acquired by the target video acquisition device. The server receives the target video stream sent by the server and processes the received target video stream according to the recreated target task. The server stores the target video stream uploaded by the target video acquisition device.

2. The method according to claim 1, characterized in that, Before the gateway device restarts, the following is also included: The server receives a task creation instruction after it first receives the target video stream, wherein the task creation instruction carries data related to the target video acquisition device; The target task is created based on the data carried by the task creation instruction; Generate the task metadata for the target task; The task metadata of the target task is stored in the data file, and the server is notified that the task creation is complete; Receive the target video stream pushed by the server, and process the target video stream according to the target task.

3. The method according to claim 2, characterized in that, The task creation instruction also carries the target type for processing the target video stream; The step of creating the target task based on the data carried by the task creation instruction includes: Based on the data carried by the task creation instruction, create a target task that processes the target video stream according to the target type; The task metadata for generating the target task includes: Generate task metadata containing the target type; The processing of the target video stream according to the target task includes: The target video stream is processed according to the processing method corresponding to the target type.

4. The method according to claim 1, characterized in that, The method further includes: Receive the notification from the server that the target video acquisition device has finished acquiring the video stream; Clear the target task and delete the task metadata of the target task stored in the data file.

5. The method according to any one of claims 1-4, characterized in that, The target task's task metadata includes the following data: The device identifier of the target video acquisition device or the task identifier of the task performed by the target video acquisition device; The video stream identifier of the target video stream; The address information of the server.

6. A service restoration device, characterized in that, Applied to a gateway device, the device includes: The data loading module is used to load the task metadata of the unfinished video processing task stored in the data file into the memory of the gateway device in response to the successful restart of the gateway device. The data file is persistently stored on the local disk, and the task metadata of the unfinished video processing task includes data related to the target video acquisition device that acquires the video stream processed by the unfinished video processing task. The task reconstruction module is used to recreate the target task for processing the target video stream acquired by the target video acquisition device based on the loaded task metadata. The first processing module is used to receive the target video stream sent by the server, process the received target video stream according to the recreated target task, and the server stores the target video stream uploaded by the target video acquisition device.

7. The apparatus according to claim 6, characterized in that, Also includes: The instruction acquisition module is used to acquire, before the gateway device restarts, a task creation instruction issued by the server after it first receives the target video stream, wherein the task creation instruction carries data related to the target video acquisition device; The task creation module is used to create the target task based on the data carried by the task creation instruction; The data generation module is used to generate the task metadata of the target task; The data storage module is used to store the task metadata of the target task into the data file and notify the server that the task creation is complete. The second processing module is used to receive the target video stream pushed by the server and process the target video stream according to the target task.

8. The apparatus according to claim 7, characterized in that, The task creation instruction also carries the target type for processing the target video stream; The task creation module is specifically used for: Based on the data carried by the task creation instruction, create a target task that processes the target video stream according to the target type; The data generation module is specifically used for: Generate task metadata containing the target type; The second processing module is specifically used for: The target video stream is processed according to the processing method corresponding to the target type.

9. The apparatus according to claim 6, characterized in that, The device further includes: The notification receiving module is used to receive the notification from the server that the video stream acquisition of the target video acquisition device has ended; The data deletion module is used to clear the target task and delete the task metadata of the target task stored in the data file.

10. The apparatus according to any one of claims 6-9, characterized in that, The target task's task metadata includes the following data: The device identifier of the target video acquisition device or the task identifier of the task performed by the target video acquisition device; The video stream identifier of the target video stream; The address information of the server.

11. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-5.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-5.

Images