A surveillance video playback system, method and storage medium for unmanned vehicle

Abstract

The present invention relates to a surveillance video playback system, method and storage medium for an unmanned vehicle, wherein the aforementioned system includes a user terminal, a signaling server, a cloud media server, a vehicle-side streaming client and a video selection forwarding server. The present invention implements the playback of surveillance videos based on the WebRTC protocol, controls the playback request time through signaling, and users can randomly order surveillance videos at any time in the past through a browser, and play back real-time streams based on time points, and users can play without downloading part or all of the video files; on the other hand, the present invention can help unmanned delivery vehicles save traffic, and unmanned delivery vehicles do not need to transmit all videos to the cloud, but pull videos on demand, and the delay in viewing the first frame is low, and the transmission efficiency is significantly improved compared to the prior art.

Description

Technical Field

[0001] The present invention relates to the field of unmanned driving technology, and in particular to a monitoring video playback system, method and storage medium for an unmanned vehicle. Background Art

[0002] With the continuous development of express delivery, e-commerce, artificial intelligence, driverless driving and other fields, many unmanned delivery vehicles have appeared in some big cities, driving on city roads, performing delivery tasks and facilitating people's lives. However, they also have certain hidden dangers, such as the loss of goods during the delivery process, accidentally being scratched by other vehicles, etc. Multiple cameras will be installed on the unmanned delivery vehicle, such as the front, rear, left and right sides, etc., to monitor and record the environment on the road, the loading and delivery of goods and other processes during its driving process without blind spots, so as to facilitate the review of past surveillance video recordings when necessary to find and clarify problems.

[0003] After starting, the delivery vehicle will record the camera video in real time and save it locally. In order to save 4G / 5G traffic and reduce operating costs, the delivery vehicle will only push the real-time video stream to the cloud when performing a delivery task. If you need to retrieve and view the historical monitoring at a certain moment, you need to check the historical video files on the vehicle. The general practice is: 1) Go to the vehicle and copy the video file directly; 2) Play back the video.

[0004] For 1), there are many ways to copy files directly from the vehicle, such as directly plugging a storage device into the vehicle, remotely logging into the vehicle, and searching and copying the corresponding files through commands; or through some LAN file sharing mechanisms. However, these operations are somewhat inconvenient and have a certain threshold for operators. They also require the support of some additional hardware or software resources, and can only be performed when the delivery vehicle is not in a driving state. It is also impossible in some special and extreme scenarios.

[0005] For 2), video playback is when the video stream is forwarded through the network, it is recorded at the same time when forwarding the cloud, and the video stream is saved in the cloud in the form of a file. When the user views the playback, he only needs to download part or all of the video file to play it. If the video to be played back is not available in the cloud, it cannot be played.

[0006] Based on this, how to design a playback system and method for unmanned vehicle surveillance videos so that users can randomly order videos from any time in the past while ensuring high-speed and low-latency transmission and playback of surveillance videos has become a technical problem that needs to be solved urgently. Summary of the invention

[0007] The invention discloses a monitoring video playback system, method and storage medium for an unmanned vehicle, aiming to solve the technical problems existing in the prior art.

[0008] The present invention adopts the following technical solutions:

[0009] In a first aspect, an embodiment of the present invention provides a surveillance video playback system for an unmanned vehicle, including a user terminal, a signaling server, a cloud media server, a vehicle-side streaming client, and a video selection and forwarding server;

[0010] The user end can initiate a playback request for a specified surveillance video to the signaling server;

[0011] The signaling server can forward the playback request to the cloud media server and / or the vehicle-side streaming client, and respond to the user end after receiving the reply;

[0012] The cloud media server can determine whether the specified surveillance video exists in the cloud based on the playback request. If so, it will push the specified surveillance video to the video selection forwarding server and reply to the signaling server; if not, it will make the signaling server forward the playback request to the vehicle-side streaming client;

[0013] The vehicle-side streaming client can determine whether the specified surveillance video exists on the vehicle side based on the playback request. If so, it will push the specified surveillance video to the video selection forwarding server and reply to the signaling server; if not, it will reply to the signaling server that the video does not exist;

[0014] The video selection forwarding server can forward the specified surveillance video to the user end.

[0015] In a possible implementation, the request parameter of the playback request includes a past time point.

[0016] In a possible implementation, the user terminal includes a user browser; the signaling server includes a WebRTC signaling server; and the video selection forwarding server includes a WebRTC protocol video selection forwarding server.

[0017] In one possible implementation, the cloud media server can store the real-time stream recorded by the unmanned vehicle when performing a task and record it in a database; based on the playback request, it can search for the video file of the specified surveillance video in the database or cache, read the file and push the parsed video stream to the WebRTC protocol video selection forwarding server.

[0018] In one possible implementation, the vehicle-side streaming client can push real-time streams and playback streams, and store camera videos on the vehicle; based on the playback request, it can search for the video file of the specified surveillance video on the vehicle side, read the file and push the parsed video stream to the WebRTC protocol video selection forwarding server.

[0019] In a possible implementation, the user browser can access the WebRTC protocol video selection forwarding server after receiving a response from the WebRTC signaling server, establish a media channel, and play the playback of the specified surveillance video.

[0020] In a second aspect, an embodiment of the present application provides a monitoring video playback method for an unmanned vehicle, the method comprising:

[0021] The user terminal initiates a playback request of a specified surveillance video to the signaling server;

[0022] The signaling server forwards the playback request to the cloud media server and / or the vehicle-side streaming client, and responds to the user end after receiving the response;

[0023] The cloud media server determines whether the specified surveillance video exists in the cloud based on the playback request. If so, it pushes the specified surveillance video to the video selection forwarding server and replies to the signaling server. If not, it enables the signaling server to forward the playback request to the vehicle-side streaming client.

[0024] The vehicle-side streaming client determines whether the specified surveillance video exists on the vehicle-side based on the playback request. If so, the specified surveillance video is pushed to the video selection forwarding server and replies to the signaling server. If not, the signaling server is replied with a message that the video does not exist.

[0025] The video selection forwarding server forwards the specified surveillance video to the user end.

[0026] In one possible implementation, the user terminal initiates a playback request for a specified surveillance video to a signaling server, and the request parameters include a past time point; the signaling server parses and obtains the past time point, and simultaneously sends the playback request and the past time point to the cloud media server and / or the vehicle-side streaming client.

[0027] In one possible implementation, the signaling server parses and obtains the past time point, and sends the playback request and the past time point to the cloud media server; the cloud media server searches for the video file of the specified surveillance video in the database or cache based on the playback request and the past time point; if it exists, the file is read and the parsed video stream is pushed to the video selection forwarding server; if it does not exist, the signaling server forwards the playback request to the vehicle-side streaming client.

[0028] In a possible implementation, the signaling server forwards the playback request and the past time point to the vehicle-side streaming client;

[0029] The vehicle-side streaming client determines whether the specified surveillance video exists on the vehicle-side based on the playback request and the past time point; if so, the specified surveillance video is pushed to the video selection forwarding server and replied to the signaling server; if not, the signaling server is replied with information that the video does not exist.

[0030] In a possible implementation, the playback request is transmitted based on the WebSocket protocol; and the specified surveillance video is transmitted based on the WebRTC protocol.

[0031] In a third aspect, the present application also provides a computer-readable storage medium, in which at least one instruction is stored, and the at least one instruction is loaded and executed by a processor to implement the monitoring video playback method of the unmanned vehicle as described above.

[0032] One embodiment of the above invention has the following advantages or beneficial effects:

[0033] The present invention mainly provides a surveillance video playback system and method for unmanned vehicles. The present invention realizes the playback of surveillance videos based on the WebRTC protocol, and controls the playback request time through signaling. Users can randomly request surveillance videos of any time in the past through a browser, and play back real-time streams based on time points. Users do not need to download part or all of the video files to achieve playback. On the other hand, the present invention can help unmanned delivery vehicles save traffic. Unmanned delivery vehicles do not need to transmit all videos to the cloud, but pull videos on demand, and the delay in viewing the first frame is low, and the transmission efficiency is significantly improved compared with the existing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments, which constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions explain the present invention and do not constitute improper limitations on the present invention. In the drawings:

[0035] Figure 1 A structural block diagram of a monitoring video playback system for an unmanned vehicle provided in one embodiment of the present invention;

[0036] Figure 2 The present invention is a flowchart of a monitoring video playback method for an unmanned vehicle provided in one embodiment of the present invention. DETAILED DESCRIPTION

[0037] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0038] When the unmanned delivery vehicle is started, it will record the camera's surveillance video in real time and save it locally. Since the video transmission is generally based on the 4G / 5G traffic provided by the IoT card, in order to save traffic, the unmanned delivery vehicle will only push the real-time video stream to the cloud when performing a task for staff to view. If you need to view historical recordings, you need to copy the recording file or play the historical recording through the video playback function.

[0039] Existing video playback is based on the download and playback of ts or fmp4 slice files from the cloud. The file-based playback results in a long delay in the user seeing the first diagnosis screen. If the video to be played back is not available in the cloud, it cannot be played. If you wait for the video to be uploaded, there will be obvious freezes in the playback, which is a very poor user experience. Furthermore, when you want to play a surveillance video after a certain historical time point, you may need to download multiple files. The first file containing the historical time point may be at a later time in the video. At this time, you still need to download the first file in its entirety, resulting in a waste of traffic.

[0040] To solve the above problems, refer to Figure 1 The embodiment of the present invention provides a surveillance video playback system for an unmanned vehicle, the system comprising a user terminal 100, a signaling server 200, a cloud media server 300, a vehicle-side streaming client 400 and a video selection and forwarding server 500; in one implementation, Figure 1 The middle dashed line represents the video media stream and WebRTC protocol stack; Figure 1 The solid lines in the middle represent the control flow and WebSocket data flow.

[0041] In an optional embodiment, the unmanned vehicle can be an unmanned delivery vehicle, an unmanned transport vehicle, an unmanned passenger vehicle, an unmanned inspection vehicle, an unmanned vending vehicle, an unmanned food delivery vehicle, an unmanned sweeper vehicle, etc.; in this embodiment, the unmanned vehicle is preferably an unmanned delivery vehicle, and one or more surveillance cameras are configured on the vehicle body to record surveillance videos of the unmanned vehicle during operation, and push real-time video streams to the cloud when performing specific delivery tasks for operators to view.

[0042] In one embodiment of the present invention, the user terminal 100 can initiate a playback request of a specified surveillance video to the signaling server 200 .

[0043] Specifically, before the user terminal 100 sends a playback request, it first determines at least that the signaling server 200 has been started to ensure that the user terminal 100 and the signaling server 200 can successfully establish a connection; since the playback request ultimately needs to be sent to the cloud media server 300 and / or the vehicle-side streaming client 400, it is more preferred that before the user terminal 100 sends a playback request, it determines that the signaling server 200, the cloud media server 300, the vehicle-side streaming client 400 and the video selection forwarding server 500 are all started to ensure that the user terminal 100 and the signaling server 200, and the signaling server 200 and the cloud media server 300 and the vehicle-side streaming client 400 can all successfully establish a connection.

[0044] In this embodiment, the user terminal 100 is specifically initiated by the operator of the unmanned vehicle to initiate a playback request. Preferably, the user terminal 100 is configured as a user browser (User Browser), so that the signaling control of the playback request and the playback of the monitoring video can be achieved through a web page (Web).

[0045] In one implementation, the playback request is transmitted based on the WebSocket protocol to achieve instant messaging; the request parameter of the playback request includes a past time point.

[0046] The HTML5 specification defines that WebSocket uses a single socket full-duplex transmission between the browser and the server to "push" and "pull" information. The data connection between the signaling server 200 and the user end 100, the signaling server 200 and the cloud media server 300 and the vehicle-side streaming client 400 is established through the WebSocket protocol during the first "handshake", and it is also based on the underlying TCP / IP protocol.

[0047] Specifically, the data transmission in the WebSocket protocol is based on frames, and the data is encoded in utf-8. Websocket enables the user's browser to achieve high efficiency and stability of web communication without installing plug-ins, so as to minimize the delay when requesting surveillance video playback.

[0048] After the playback request is finally sent to the cloud media server 300 or the vehicle-side streaming client 400 via the signaling server 200, the cloud media server 300 or the vehicle-side streaming client 400 can push the playback video stream to the selected forwarding server and reply to the signaling server 200. After receiving the reply, the signaling server 200 responds to the user terminal 100. After receiving the reply from the signaling server 200, the user terminal 100 can access the video selection forwarding server 500, establish a media channel, and play the playback of the specified surveillance video.

[0049] In one embodiment of the present invention, the signaling server 200 can forward the playback request to the cloud media server 300 and / or the vehicle-side streaming client 400, and respond to the user end 100 after receiving a reply.

[0050] In this embodiment, the signaling server 200 is a WebRTC signaling server (WebRTC Signaling Server) to achieve low latency in milliseconds; WebRTC is also known as Web Real-Time Communication, which stands for network real-time communication. WebRTC signaling is the process of establishing, controlling and terminating communication sessions between peers. The WebRTC signaling server is a server that manages connections between peers. It is only used for signaling, which helps one peer find another peer in the network, negotiate the connection itself, reset the connection when necessary, and then close it. All peers send notifications to the WebRTC signaling server that they are connected, and the server forwards them to the peers. Notification requests can be completed through WebSocket or HTTP protocols.

[0051] In one implementation, the user end 100 and the signaling server 200 , the signaling server 200 and the cloud media server 300 , and the vehicle-side streaming client 400 are all connected via WebSocket.

[0052] In one implementation, the signaling server 200 can parse and obtain the time point in the playback request, and send the playback request, signaling and time point to the cloud media server 300 and the vehicle-side streaming client 400 at the same time, and respond to the user end 100 after receiving the first reply.

[0053] In one implementation, the signaling server 200 can parse and obtain the time point in the playback request, and send the playback request, signaling and time point to the cloud media server 300, and wait for the reply of the cloud media server 300. If the video file of the specified surveillance video exists in the cloud database or cache, the signaling server 200 will receive an affirmative reply (or the file exists), and the signaling server 200 will respond to the user end 100; if the video file of the specified surveillance video does not exist in the cloud database or cache, the signaling server 200 will receive a negative reply (or The signaling server 200 will receive a negative (or incorrect / file does not exist) reply, and the signaling server 200 will forward the playback request and the time point to the vehicle-side streaming client 400. If the video file of the specified surveillance video exists on the vehicle-side, the signaling server 200 will receive an affirmative (or file exists) reply, and the signaling server 200 will respond to the user end 100; if the video file of the specified surveillance video does not exist on the vehicle-side, the signaling server 200 will receive a negative (or incorrect / file does not exist) reply, and the signaling server 200 will respond to the user end 100 that the specified surveillance video is not found.

[0054] In one embodiment of the present invention, the cloud media server 300 can determine whether the specified surveillance video exists in the cloud based on the playback request. If so, the specified surveillance video will be pushed to the video selection forwarding server 500 and reply to the signaling server 200; if not, the signaling server 200 will forward the playback request to the vehicle-side streaming client 400.

[0055] Specifically, there is a connection between the cloud media server 300 (Media Server) and the vehicle-side streaming client 400. The cloud media server 300 can store the real-time video stream recorded by the unmanned vehicle when performing a task and record it in a database.

[0056] In one implementation, the cloud media server 300 and the vehicle-side streaming client 400 realize video stream transmission through the video selection forwarding server 500. On the one hand, the real-time video stream provided by the vehicle-side streaming client 400 is forwarded to the user terminal 100 through the video selection forwarding server 500 for real-time playback by the user terminal 100; on the other hand, the real-time video stream provided by the vehicle-side streaming client 400 is forwarded to the cloud media server 300 through the video selection forwarding server 500 for storage and recording.

[0057] In this embodiment, the cloud media server 300 is able to receive a playback request and time point from the signaling server 200, search for the video file of the specified surveillance video in the database or cache, read the file and push the parsed video stream to the video selection forwarding server 500, and reply to the signaling server 200 at the same time; if the corresponding video file is not stored in the database or cache, it will reply to the signaling server 200 that the file does not exist. At this time, the signaling server 200 will continue to send the signaling and time point of the playback request to the vehicle-side streaming client 400.

[0058] In one embodiment of the present invention, the vehicle-side streaming client 400 (Vehicle Client) can determine whether the specified surveillance video exists on the vehicle-side based on the playback request. If so, the specified surveillance video will be pushed to the video selection forwarding server 500 and replied to the signaling server 200; if not, the signaling server 200 will be replied with information that the video does not exist.

[0059] In one implementation, the vehicle-side streaming client 400 is data-connected to a video acquisition device, such as surveillance cameras arranged at various locations on the vehicle body; on the one hand, the vehicle-side streaming client 400 can acquire video streams and push real-time streams and playback streams to a video selection forwarding server 500; on the other hand, the vehicle-side streaming client 400 can also store camera recordings in a vehicle-side memory.

[0060] In this embodiment, the vehicle-side streaming client 400 can search for the video file of the specified surveillance video in the vehicle-side memory based on the signaling and time point of the playback request, read the file and push the parsed video stream to the video selection forwarding server 500, and reply to the signaling server 200 at the same time; if the vehicle-side does not store the corresponding video file, it will reply to the signaling server 200 with the information that the file does not exist. At this time, the signaling server 200 will respond to the user end 100 with the information that the specified surveillance video does not exist.

[0061] In one embodiment of the present invention, the video selection forwarding server 500 is capable of forwarding the designated surveillance video to the user terminal 100 .

[0062] In this embodiment, the video selective forwarding server 500 is a WebRTC protocol video selective forwarding server (WebRTCSFU), wherein the full name of SFU is Selective Forwarding Unit, which is a server-side program for routing and forwarding WebRTC client audio and video data streams. The core function of WebRTC SFU is to establish a link with each WebRTC Peer client, receive audio and video data from them respectively, and realize the one-to-many capability (i.e. forwarding a client stream to other WebRTC Peer clients). In this embodiment, that is, WebRTC SFU simultaneously establishes a connection between the user end 100 and the vehicle-side streaming client 400 or the user end 100 and the cloud media server 300, the vehicle-side streaming client 400 or the cloud media server 300 can push the video stream to WebRTC SFU, the user end 100 accesses WebRTC SFU, establishes a media channel, and plays the playback video.

[0063] In one embodiment of the present invention, the above-mentioned monitoring video playback system for unmanned vehicles performs the following steps when in use:

[0064] The WebRTC signaling server, the cloud media server 300 and the WebRTC protocol video selection forwarding server are all started to ensure that connections are successfully established between various endpoints in the system;

[0065] The operator logs in to the user browser as the client 100, initiates a playback request of the specified surveillance video to the WebRTC signaling server through the user browser, and waits for a response; at this time, the parameters of the playback request include a past time point;

[0066] After receiving the playback request, the WebRTC signaling server parses and obtains the time point, and forwards the signaling of the playback request and the aforementioned past time point to the cloud media server 300;

[0067] The cloud media server 300 searches the cloud media library or cache to check whether the corresponding video file of the specified surveillance video is stored;

[0068] If the cloud media server 300 stores the video file, it reads the file and pushes the parsed video stream to the WebRTC protocol video selection forwarding server, and replies to the WebRTC signaling server. After receiving the reply, the WebRTC signaling server replies to the user terminal 100. The user terminal 100 accesses the WebRTC protocol video selection forwarding server, establishes a media channel, and plays the playback video;

[0069] If the video file does not exist in the cloud media server 300, the WebRTC signaling server forwards the playback request to the vehicle-side streaming client 400;

[0070] The vehicle-side streaming client 400 searches for the video file in the vehicle-side storage. If the video file is stored in the vehicle-side, the file is read and the parsed video stream is pushed to the WebRTC protocol video selection forwarding server, and the WebRTC signaling server is replied. After receiving the reply, the WebRTC signaling server replies to the user end 100. The user end 100 accesses the WebRTC protocol video selection forwarding server, establishes a media channel, and plays the playback video.

[0071] If the vehicle does not store the video file, it replies to the WebRTC signaling server that the video file of the specified surveillance video is not found.

[0072] Please refer to Figure 2 , which shows a method flow chart of a monitoring video playback method for an unmanned vehicle provided by an embodiment of the present application. The monitoring video playback method is preferably implemented based on the above-mentioned monitoring video playback system for the unmanned vehicle; the monitoring video playback method can enable the user's browser to randomly play the recording of the unmanned vehicle at any time in the past, and play back the real-time stream based on the time point, without the user having to download the file.

[0073] The monitoring video playback method of the unmanned vehicle may include:

[0074] Step 201: The user terminal initiates a playback request for a specified surveillance video to a signaling server.

[0075] Specifically, before executing step 201, first at least determine that the signaling server 200 has been started to ensure that the user terminal 100 and the signaling server 200 can successfully establish a connection; since the playback request ultimately needs to be sent to the cloud media server 300 and / or the vehicle-side streaming client 400, it is more preferred that before the user terminal 100 sends the playback request, it is determined that the signaling server 200, the cloud media server 300, the vehicle-side streaming client 400 and the video selection forwarding server 500 are all started to ensure that the user terminal 100 and the signaling server 200, and the signaling server 200 and the cloud media server 300 and the vehicle-side streaming client 400 can all successfully establish a connection.

[0076] Specifically, the user terminal 100 is specifically initiated by the operator of the unmanned vehicle to initiate a playback request. Preferably, the user terminal 100 is configured as a user browser (User Browser), so that the signaling control of the playback request and the playback of the monitoring video can be achieved through a web page (Web).

[0077] In one implementation, the playback request is transmitted based on the WebSocket protocol to achieve instant messaging; the request parameter of the playback request for the specified surveillance video includes a past time point.

[0078] Step 202: The signaling server forwards the playback request to the cloud media server and / or the vehicle-side streaming client, and responds to the user end after receiving a response.

[0079] Specifically, the signaling server 200 is a WebRTC signaling server (WebRTC Signaling Server).

[0080] In one implementation, the signaling server 200 can parse and obtain the time point in the playback request, and send the playback request, signaling and time point to the cloud media server 300, and wait for the reply of the cloud media server 300. If the video file of the specified surveillance video exists in the cloud database or cache, the signaling server 200 will receive an affirmative reply (or the file exists), and the signaling server 200 will respond to the user end 100; if the video file of the specified surveillance video does not exist in the cloud database or cache, the signaling server 200 will receive a negative reply (or The signaling server 200 will receive a negative (or incorrect / file does not exist) reply, and the signaling server 200 will forward the playback request and the time point to the vehicle-side streaming client 400. If the video file of the specified surveillance video exists on the vehicle-side, the signaling server 200 will receive an affirmative (or file exists) reply, and the signaling server 200 will respond to the user end 100; if the video file of the specified surveillance video does not exist on the vehicle-side, the signaling server 200 will receive a negative (or incorrect / file does not exist) reply, and the signaling server 200 will respond to the user end 100 that the specified surveillance video is not found.

[0081] Step 203. The cloud media server determines whether the specified surveillance video exists in the cloud based on the playback request. If so, the specified surveillance video is pushed to the video selection forwarding server and replies to the signaling server; if not, the signaling server forwards the playback request to the vehicle-side streaming client.

[0082] Specifically, the cloud media server 300 can receive the playback request and time point from the signaling server 200, search for the video file of the specified surveillance video in the database or cache, read the file and push the parsed video stream to the video selection and forwarding server 500. The video selection and forwarding server 500 is preferably a WebRTC protocol video selection and forwarding server, and the video stream is transmitted based on the WebRTC protocol; the remote media server replies to the signaling server 200 at the same time; if the corresponding video file is not stored in the database or cache, it replies to the signaling server 200 that the file does not exist. At this time, the signaling server 200 will continue to send the signaling and time point of the playback request to the vehicle-side streaming client 400.

[0083] Step 204. The vehicle-side streaming client determines whether the specified surveillance video exists on the vehicle-side based on the playback request. If so, the specified surveillance video is pushed to the video selection forwarding server and replied to the signaling server; if not, the signaling server is replied with information that the video does not exist.

[0084] Specifically, the vehicle-side streaming client 400 can search for the video file of the specified surveillance video in the vehicle-side storage based on the signaling and time point of the playback request, read the file and push the parsed video stream to the WebRTC protocol video selection forwarding server, and reply to the signaling server 200 at the same time; if the vehicle-side does not store the corresponding video file, it will reply to the signaling server 200 with the information that the file does not exist. At this time, the signaling server 200 will respond to the user end 100 with the information that the specified surveillance video does not exist.

[0085] Step 205: The video selection forwarding server forwards the designated surveillance video to the user end.

[0086] Specifically, after receiving an affirmative response from the WebRTC signaling server, the user terminal 100 accesses the WebRTC protocol video selection forwarding server, establishes a media channel, and plays the playback video.

[0087] The surveillance video playback method of the unmanned vehicle described in the above embodiment realizes the playback of surveillance videos based on the WebRTC protocol, and controls the playback request time through signaling. The user can randomly request surveillance videos of any time in the past through the browser, and replay the real-time stream based on the time point. The user does not need to download part or all of the video files to achieve playback; on the other hand, the above embodiment can help unmanned delivery vehicles save traffic. The unmanned delivery vehicle does not need to transmit all videos to the cloud, but pulls videos on demand, and the delay in viewing the first frame is low, and the transmission efficiency is significantly improved compared with the existing technology.

[0088] An embodiment of the present application provides a computer-readable storage medium, in which at least one instruction is stored. The at least one instruction is loaded and executed by a processor to implement the monitoring video playback method of the unmanned vehicle as described above.

[0089] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0090] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A surveillance video playback system for an unmanned vehicle, characterized in that: It includes the user end, signaling server, cloud media server, vehicle-side streaming client and video selection and forwarding server; The user terminal can initiate a playback request of a specified surveillance video to the signaling server; The signaling server is capable of forwarding the playback request to the cloud media server and / or the vehicle-side streaming client, and responding to the user end after receiving a reply; The cloud media server can store the real-time video stream recorded by the unmanned vehicle when performing tasks and record it in the database; The cloud media server can also determine whether the specified surveillance video exists in the cloud based on the playback request, and if so, push the specified surveillance video to the video selection forwarding server and reply to the signaling server; If not, the signaling server forwards the playback request to the vehicle-side streaming client; The vehicle-side streaming client can determine whether the specified monitoring video exists on the vehicle side based on the playback request. If so, the specified monitoring video is pushed to the video selection forwarding server and replied to the signaling server; If the video does not exist, a message indicating that the video does not exist is sent to the signaling server; The video selection forwarding server can simultaneously establish a connection between the user end and the vehicle-end streaming client or the user end and the cloud media server. The vehicle-end streaming client or the cloud media server can push the video stream to the video selection forwarding server, and the video selection forwarding server can forward the specified surveillance video to the user end.

2. The monitoring video playback system for an unmanned vehicle according to claim 1, characterized in that: The request parameter of the playback request includes a past time point.

3. The monitoring video playback system for an unmanned vehicle according to claim 1, characterized in that: The user terminal includes a user browser; the signaling server includes a WebRTC signaling server; and the video selection and forwarding server includes a WebRTC protocol video selection and forwarding server.

4. The monitoring video playback system for an unmanned vehicle according to claim 3, characterized in that: The cloud media server can store the real-time stream recorded by the unmanned vehicle when performing a task and record it in a database; based on the playback request, it can search for the video file of the specified surveillance video in the database or cache, read the file and push the parsed video stream to the WebRTC protocol video selection forwarding server.

5. The monitoring video playback system for an unmanned vehicle according to claim 3, characterized in that: The vehicle-side streaming client can push real-time streams and playback streams, and store camera videos on the vehicle; based on the playback request, it can search for the video file of the specified surveillance video on the vehicle side, read the file and push the parsed video stream to the WebRTC protocol video selection forwarding server.

6. The monitoring video playback system for an unmanned vehicle according to claim 3, characterized in that: The user browser can access the WebRTC protocol video selection forwarding server after receiving a response from the WebRTC signaling server, establish a media channel, and play the playback of the specified monitoring video.

7. A monitoring video playback method for an unmanned vehicle, characterized in that: The method comprises: The user terminal initiates a playback request of a specified surveillance video to the signaling server; The signaling server forwards the playback request to the cloud media server and / or the vehicle-side streaming client, and responds to the user end after receiving a reply; The cloud media server can store the real-time video stream recorded by the unmanned vehicle when performing the task and record it in the database; the cloud media server also determines whether the specified monitoring video exists in the cloud based on the playback request. If so, the specified monitoring video is pushed to the video selection forwarding server and replied to the signaling server; if not, the signaling server forwards the playback request to the vehicle-side streaming client; The vehicle-side streaming client determines whether the specified surveillance video exists on the vehicle-side based on the playback request. If so, the specified surveillance video is pushed to the video selection forwarding server and replies to the signaling server; if not, the signaling server is replied with a message that the video does not exist. The video selection forwarding server simultaneously establishes a connection between the user end and the vehicle-end streaming client or the user end and the cloud media server. The vehicle-end streaming client or the cloud media server pushes the video stream to the video selection forwarding server, and the video selection forwarding server forwards the specified surveillance video to the user end.

8. The monitoring video playback method of the unmanned vehicle according to claim 7, characterized in that: The method comprises: The user terminal initiates a playback request of a specified surveillance video to the signaling server, and the request parameter includes a past time point; The signaling server parses and obtains the past time point, and sends the playback request and the past time point to the cloud media server and / or the vehicle-side streaming client.

9. The monitoring video playback method of the unmanned vehicle according to claim 8, characterized in that: The method comprises: The signaling server parses and obtains the past time point, and sends the playback request and the past time point to the cloud media server; Based on the playback request and the past time point, the cloud media server searches for the video file of the specified surveillance video in the database or cache; if it exists, it reads the file and pushes the parsed video stream to the video selection forwarding server; if it does not exist, it enables the signaling server to forward the playback request to the vehicle-side streaming client.

10. The monitoring video playback method of an unmanned vehicle according to claim 9, characterized in that: The method comprises: The signaling server forwards the playback request and the past time point to the vehicle-side streaming client; The vehicle-side streaming client determines whether the specified surveillance video exists on the vehicle-side based on the playback request and the past time point; if so, the specified surveillance video is pushed to the video selection forwarding server and replied to the signaling server; if not, the signaling server is replied with information that the video does not exist.

11. The monitoring video playback method of an unmanned vehicle according to any one of claims 7 to 10, characterized in that: The playback request is transmitted based on the WebSocket protocol; the specified monitoring video is transmitted based on the WebRTC protocol.

12. A computer-readable storage medium, characterized in that: At least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by the processor to implement the monitoring video playback method of the unmanned vehicle as described in any one of claims 7-11.

Images