Network video surveillance system

Abstract

The invention discloses a network video monitoring system, which receives corresponding multiple video stream data pushed by multiple front-end devices through a video stream receiving module and transmits the multiple video stream data; the video stream real-time preview module sends the received multiple video stream data to multiple monitoring mobile terminals for real-time preview; the video stream playback recording module processes the received multiple video stream data to obtain video files for playback; the unified storage module backs up and stores the received video files; the video stream processing module identifies and processes the multiple video stream data and stores the obtained identification results; the video stream distribution module transmits the multiple video stream data to a content distribution network through a real-time message transmission protocol for online viewing; thus, multifunctional video monitoring is realized with low cost and high reliability by transmitting, storing and managing the video streams of monitoring cameras on a video monitoring cloud platform.

Description

Technical Field

[0001] The present invention relates to the technical field of video data processing, and in particular to a network video monitoring system. Background Art

[0002] In the related technologies, video surveillance is an important part of the security system. It is widely used in many occasions because of its intuitiveness, accuracy, timeliness and rich information content. In recent years, with the rapid development of computers, networks, image processing and transmission technologies, video surveillance technology has also made great progress. At present, the field of video surveillance widely uses network video recorders (NVR) to store and process surveillance videos. However, this method requires the purchase of NVR equipment, regular maintenance after installation, the use of hard disk storage and lack of reasonable redundant configuration, which leads to problems such as large investment, high operation and maintenance costs, low storage reliability and single function and poor integration. Summary of the invention

[0003] The present invention aims to solve at least one of the technical problems in the above-mentioned technologies to a certain extent. To this end, one object of the present invention is to provide a network video monitoring system, which can realize multifunctional video monitoring with low cost and high reliability by transmitting, storing and managing the video stream of the monitoring camera on a video monitoring cloud platform, thereby greatly improving the user experience.

[0004] To achieve the above-mentioned purpose, an embodiment of the present invention proposes a network video monitoring system, comprising: multiple monitoring mobile terminals, a video monitoring cloud platform and multiple front-end devices, wherein the video monitoring cloud platform is connected to the multiple monitoring mobile terminals and the multiple front-end devices respectively, and the video monitoring cloud platform comprises a video stream receiving module, a video stream real-time preview module, a video stream playback recording module, a unified storage module, a video stream processing module and a video stream distribution module; wherein the video stream receiving module is used to receive the corresponding multiple video stream data pushed by the multiple front-end devices and transmit the multiple video stream data; the video stream real-time preview module is used to send the received multiple video stream data to the multiple monitoring mobile terminals so that the multiple monitoring mobile terminals can The multiple video stream data are previewed in real time; the video stream playback and recording module is used to process the received multiple video stream data to obtain a video file so that the multiple monitoring mobile terminals can play back the video file; the unified storage module is used to back up and store the received video file so that the multiple monitoring mobile terminals can retrieve the backed-up video file; the video stream processing module is used to generate thumbnails, detect motion and identify objects for the multiple video stream data, and store the obtained identification results so that the multiple monitoring mobile terminals can retrieve them; the video stream distribution module is used to transmit the multiple video stream data to a content distribution network through a real-time messaging protocol so that the multiple monitoring mobile terminals can watch them online.

[0005] According to the network video monitoring system of the embodiment of the present invention, a video stream receiving module is set to receive the corresponding multiple video stream data pushed by multiple front-end devices and transmit the multiple video stream data; a video stream real-time preview module is used to send the received multiple video stream data to multiple monitoring mobile terminals, so that the multiple monitoring mobile terminals can preview the multiple video stream data in real time; a video stream playback and recording module is used to process the received multiple video stream data to obtain a video file so that the multiple monitoring mobile terminals can play back the video file; a unified storage module is used to back up and store the received video file so that the multiple monitoring mobile terminals can retrieve the backed-up video file; a video stream processing module is used to generate thumbnails, detect motion and identify objects for the multiple video stream data, and store the obtained identification results so that the multiple monitoring mobile terminals can retrieve them; a video stream distribution module is used to transmit the multiple video stream data to a content distribution network through a real-time message transmission protocol so that the multiple monitoring mobile terminals can watch online; thus, by transmitting, storing and managing the video streams of monitoring cameras on a video monitoring cloud platform, multifunctional video monitoring is achieved with low cost and high reliability, thereby greatly improving user experience.

[0006] In addition, the network video monitoring system proposed in the above embodiment of the present invention may also have the following additional technical features:

[0007] Optionally, the multiple front-end devices are cameras supporting different formats.

[0008] Optionally, the video stream receiving module includes an RTMP module, an SRT module and a GB28181 module. After the multiple cameras send their own device numbers, supported video formats, and encoding information to the signaling service to complete device registration, the video stream receiving module returns the RTMP / SRT endpoint address, and for the GB28181 protocol, returns SIP registration information to the corresponding multiple cameras, so that the multiple cameras can push their respective video stream data to the module corresponding to the video stream receiving module.

[0009] Optionally, the video stream real-time preview module includes an HLS module, a WebRTC module and a DASH module, wherein the HLS module is used to provide good compatibility for the multiple monitoring mobile terminals, the WebRTC module is used to provide ultra-low latency monitoring real-time viewing, and the DASH module is used to divide each of the multiple video stream data into multiple file fragments with multiple bit rates, so that each of the multiple monitoring mobile terminals automatically selects to download and play a corresponding file fragment according to current network conditions.

[0010] Optionally, the video stream playback and recording module includes a real-time playback module, a historical playback module and a playback export module, wherein the real-time playback module adopts a long-window HLS live broadcast to provide near-line playback, the historical playback module dynamically generates an HLS index file as needed to provide historical playback, and the playback export module provides an encapsulated FLV format recording file for exporting playback.

[0011] Optionally, the unified storage module includes a tiered storage module, a quota judgment module and a periodic settlement module. The tiered storage module uses S3 or S3-compatible to perform tiered storage on the video files. The quota judgment module is used to limit the available capacity of the tiered storage module according to preset conditions, or to limit the total available storage of a certain monitoring mobile terminal according to the configuration, and to provide two optional modes of alarm or overwrite when the limit is full / exceeded. The periodic settlement module is used to periodically settle the video file backup from hot storage to a low-frequency / archive access type according to the settings, and to thaw the archive storage when the monitoring mobile terminal retrieves the backup video file.

[0012] Optionally, the video stream processing module includes a motion detection module, a thumbnail generation module and an object recognition module, wherein the motion detection module is used to perform frame extraction processing on each of the multiple video stream data and judge the motion of the picture according to the image through a frame difference method or a background segmentation method to obtain a motion detection result, the thumbnail generation module is used to extract frames for each video stream data according to a fixed interval to generate a thumbnail and write it into storage, and the object recognition module is used to implement object recognition through the Google TensorFlow&DeepStack artificial intelligence suite to obtain a recognition result.

[0013] Optionally, the monitoring mobile terminal may be any one of a mobile phone, a computer or a tablet.

[0014] Optionally, the network video monitoring system further includes a notification and alarm module, which is used to receive notification information generated by each module and push the notification information to the corresponding monitoring mobile terminal according to the contact information and channel configuration corresponding to the notification information.

[0015] Optionally, the network video monitoring system also includes an API application program interface so that the monitoring mobile terminal can retrieve relevant data. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A block diagram of a network video monitoring system according to an embodiment of the present invention;

[0017] Figure 2 A block diagram of a video stream receiving module according to an embodiment of the present invention;

[0018] Figure 3 A block diagram of a video stream real-time preview module according to an embodiment of the present invention;

[0019] Figure 4 A block diagram of a video stream playback and recording module according to an embodiment of the present invention;

[0020] Figure 5 is a block diagram of a unified storage module according to an embodiment of the present invention;

[0021] Figure 6 is a block diagram of a video stream processing module according to an embodiment of the present invention;

[0022] Figure 7 FIG. 4 is a block diagram of a video stream distribution module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0024] In the related technologies, network video recorders are widely used in the field of video surveillance to store and process surveillance videos. However, this method requires the purchase of NVR equipment and hard disks, which requires scenario analysis and the formulation of a purchase plan, resulting in high equipment investment. After installation, in order to ensure stable operation, a dedicated person is required to be responsible for daily maintenance and updates, exception handling, etc., which increases labor costs. NVR equipment uses hard disks as storage media, and there are many reasons that affect storage reliability, such as: sudden failure of the hard disk, failure to configure RAID1 or above redundant storage arrays, configuration of RAID but a single disk is damaged, a large amount of reading and writing during the reconstruction of the storage array causes the backup disk to be damaged again, and the same batch of products purchased with the same problem fail at the same time. The functions provided by NVR are relatively single, mostly video backup and playback, which limits the application scope of video surveillance and cannot be well integrated with surrounding systems.

[0025] To this end, the present invention proposes a network video monitoring system, which is configured to receive a plurality of corresponding video stream data pushed by a plurality of front-end devices and transmit the plurality of video stream data; a video stream real-time preview module is configured to send the received plurality of video stream data to a plurality of monitoring mobile terminals, so that the plurality of monitoring mobile terminals can preview the plurality of video stream data in real time; a video stream playback and recording module is configured to process the received plurality of video stream data to obtain a video file, so that the plurality of monitoring mobile terminals can play back the video file; a unified storage module is configured to back up and store the received video file, so that the plurality of monitoring mobile terminals can retrieve the backed-up video file; a video stream processing module is configured to generate thumbnails, perform motion detection and object recognition on the plurality of video stream data, and store the obtained recognition results, so that the plurality of monitoring mobile terminals can retrieve the same; a video stream distribution module is configured to transmit the plurality of video stream data to a content distribution network through a real-time message transmission protocol, so that the plurality of monitoring mobile terminals can watch the video online; thus, by transmitting, storing and managing the video streams of monitoring cameras on a video monitoring cloud platform, multifunctional video monitoring is realized with low cost and high reliability, thereby greatly improving the user experience.

[0026] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0027] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0028] Figure 1 FIG. 4 is a block diagram of a network video monitoring system according to an embodiment of the present invention; Figure 1 As shown, the network video surveillance system includes multiple monitoring mobile terminals 1, a video surveillance cloud platform 2 and multiple front-end devices 3.

[0029] Among them, the video surveillance cloud platform 2 is connected to multiple monitoring mobile terminals 1 and multiple front-end devices 3 respectively, and the video surveillance cloud platform 2 includes a video stream receiving module 21, a video stream real-time preview module 22, a video stream playback recording module 23, a unified storage module 24, a video stream processing module 25 and a video stream distribution module 26.

[0030] The video stream receiving module 21 is used to receive corresponding multiple video stream data pushed by multiple front-end devices 3, and transmit the multiple video stream data.

[0031] As an embodiment, the multiple front-end devices 3 are cameras supporting different formats.

[0032] As an example, Figure 2 As shown, the video stream receiving module 21 includes an RTMP module 211, an SRT module 212 and a GB28181 module 213. After multiple cameras send their own device numbers, supported video formats, and encoding information to the signaling service to complete device registration, the video stream receiving module returns the RTMP / SRT endpoint address, and for the GB28181 protocol, returns SIP (Session initialization Protocol) registration information to the corresponding multiple cameras, so that the multiple cameras can push their respective video stream data to the module corresponding to the video stream receiving module 21.

[0033] That is to say, the video stream receiving module 21 supports the transmission of camera video streams through RTMP (Real-Time Messaging Protocol), SRT (Secure Reliable Transport) and GB28181 (national standard); RTMP provides extremely strong compatibility with most devices, and using SRT for supported devices further reduces latency; camera A that supports RTMP, camera B that supports GB28181, and camera C that supports SRT complete device registration by sending their own device numbers, supported video formats, encodings and other information to the signaling service; the video stream receiving module 21 returns the RTMP endpoint address to camera A, returns the SRT endpoint address to camera C, and returns the SIP registration information to camera B; so that camera A pushes the video stream to the RTMP module 211, camera B pushes the video stream to the GB28181 module 213, and camera C pushes the video stream to the SRT module 212.

[0034] The video stream real-time preview module 22 is used to send the received multiple video stream data to multiple monitoring mobile terminals 1, so that the multiple monitoring mobile terminals 1 can preview the multiple video stream data in real time.

[0035] As an example, Figure 3 As shown, the video stream real-time preview module 22 includes an HLS module 221, a WebRTC module 222 and a DASH module 223, wherein the HLS module 221 is used to provide good compatibility for multiple monitoring mobile terminals 1, the WebRTC module 222 is used to provide ultra-low latency monitoring real-time viewing, and the DASH module 223 is used to divide each of the multiple video stream data into multiple file segments with multiple bit rates, so that each of the multiple monitoring mobile terminals 1 automatically selects to download and play a corresponding file segment according to the current network conditions.

[0036] That is to say, the video stream real-time preview module 22 supports low-latency monitoring and viewing in FLV format (delay of about 2s); at the same time, it provides ultra-low latency (delay of about 500ms) monitoring and real-time viewing through WebRTC (Web Real-Time Communications); adaptive bit rate DASH (Dynamic Adaptive Streaming over HTTP) supports dividing the live stream into multiple file segments with multiple bit rates, and the monitoring mobile terminal 1 will automatically choose which segment to download and play according to the current network conditions; when the network conditions change, such as the available bandwidth decreases and the packet loss rate increases, the monitoring mobile terminal 1 will download the low bit rate version as the next segment, so as to avoid the impact of disconnection on the experience, thereby seamlessly adapting to the changing network conditions and providing a high-quality playback experience; HLS (HTTP Live Streaming) provides good compatibility for mobile devices.

[0037] It should be noted that the video stream real-time preview module 22 can provide multiple forms of web versions and Apps. It provides a browser-based user interface for desktop systems such as Windows, Linux, and macOS, and a browser-based user interface (adaptive web page) for mobile devices such as Android and iOS. At the same time, Apps are provided for installation and use, and there is no difference in the functions provided by the two.

[0038] The video stream playback and recording module 23 is used to process the received multiple video stream data to obtain a video file so that multiple monitoring mobile terminals 1 can play back the video file.

[0039] As an example, Figure 4 As shown, the video stream playback and recording module 23 includes a real-time playback module 231, a historical playback module 232 and a playback export module 233, wherein the real-time playback module 231 adopts a long-window HLS live broadcast to provide near-line playback, the historical playback module 232 dynamically generates an HLS index file as needed to provide historical playback, and the playback export module 233 provides a packaged FLV format video file for exporting playback.

[0040] It should be noted that long-window HLS live broadcast provides near-line playback without waiting for video cutting; HLS index files are dynamically generated as needed to provide historical playback; when playback needs to be exported, packaged FLV format video files are provided; transcoding and slicing are performed into the H264 universal format for short-term playback; efficient encoding is used to generate video clip files for archiving, where video encoding algorithms used include but are not limited to: H.264, H.265 (HEVC), AV1, VP9, ​​and audio encoding used include but are not limited to: AAC, OPUS.

[0041] The unified storage module 24 is used to back up and store the received video files so that multiple monitoring mobile terminals 1 can retrieve the backed-up video files.

[0042] As an example, Figure 5 As shown, the unified storage module 24 includes a tiered storage module 241, a quota determination module 242 and a periodic settlement module 243. The tiered storage module 241 uses S3 (Simple Storage Service) object storage or S3-compatible (S3-compatible object storage) to store video files in tiers. The quota determination module 242 is used to limit the available capacity of the tiered storage module according to preset conditions, or to limit the total available storage of a certain monitoring mobile terminal 1 according to the configuration, and to provide two optional modes of alarm or overwrite when the limit is full / exceeded. The periodic settlement module 243 is used to regularly settle the video file backup from hot storage to low-frequency / archive access type according to the settings, and to thaw the archive storage when receiving the video file retrieved by the monitoring mobile terminal 1.

[0043] It should be noted that hot storage is standard object storage; the characteristics of object storage determine that archive storage needs to be thawed in advance when it is called. After receiving the retrieval request, the periodic sedimentation module 243 automatically selects the extreme speed (Explicit) and normal speed to thaw the file according to the file size, and provides friendly prompts. When the video playback progress is close to the archive storage file block, the periodic sedimentation module 243 will thaw in advance to reduce the user's perception of configuring the life cycle for the thawed file. The periodic sedimentation module 243 continues to settle as archive storage after a specified number of days.

[0044] The video stream processing module 25 is used to generate thumbnails, detect motion and identify objects for multiple video stream data, and store the obtained identification results so that multiple monitoring mobile terminals can retrieve them.

[0045] As an example, Figure 6 As shown, the video stream processing module 25 includes a motion detection module 251, a thumbnail generation module 252 and an object recognition module 253, wherein the motion detection module 251 is used to perform frame extraction processing on each video stream data in multiple video stream data and judge the picture motion according to the image through the frame difference method (Temporal Difference) or the background subtraction method (Background subtraction) to obtain the motion detection result, the thumbnail generation module 252 is used to extract the frame of each video stream data according to a fixed interval to generate a thumbnail and write it into the storage, and the object recognition module 253 is used to realize object recognition through artificial intelligence suites such as Google TensorFlow&DeepStack to obtain the recognition result.

[0046] That is to say, the motion detection module 251 extracts frames from the video and judges the motion of the picture according to the image through algorithms such as the frame difference method and the background segmentation method (supports setting a detection area mask to ignore changes in the specified area); the thumbnail generation module 252 extracts frames from the video according to a fixed interval (the frame extraction results in the motion detection can be reused), generates thumbnails, and writes them into the storage module. It should be noted that this step is also combined with the motion detection results. When there are changes in the pictures between adjacent frames, one or more frames with changes are preferentially extracted to optimize the representativeness of the thumbnails and improve the playback efficiency; the object recognition module 253 includes the recognition and analysis of pedestrians and common objects, and realizes object recognition through artificial intelligence suites such as Google TensorFlow&DeepStack.

[0047] The video stream distribution module 26 is used to transmit multiple video stream data to a content distribution network through a real-time message transmission protocol so that multiple monitoring mobile terminals can watch online.

[0048] As an example, Figure 7 As shown, in a high-traffic scenario, the video stream distribution module 26 (central server) requires a large bandwidth to simultaneously transmit multiple video streams. Taking one video at 2Mbps as an example, assuming that at a certain moment there are 100 cameras on the platform being previewed in real time, and each camera is simultaneously watched by 10 monitoring mobile terminals, this requires the central server to have an uplink bandwidth of 2*100*10=2Gbps. The bandwidth pressure of the central server can be significantly reduced through the content delivery network (CDN).

[0049] That is to say, the central server transmits the video stream to the content distribution network through RTMP, ensuring low latency. No matter how many monitoring mobile terminals watch a single video stream, the central server always has only one video stream output. Under this structure, the bandwidth consumption of the central server is 2*100=200Mbps=0.2Gbps, which is 10% of the original solution.

[0050] As an embodiment, the monitoring mobile terminal may be any one of a mobile phone, a computer or a tablet; the present invention does not make any specific limitation to this.

[0051] As an embodiment, the network video monitoring system further includes a notification and alarm module, which is used to receive notification information generated by each module and push the notification information to the corresponding monitoring mobile terminal according to the contact information and channel configuration corresponding to the notification information.

[0052] In other words, the network video surveillance system supports notification services through multiple channels (SMS, WeChat, email, Telegram, Web Hook, etc.), receives information generated by each module, and pushes it to the recipient according to contact information and channel configuration.

[0053] As an embodiment, the network video monitoring system also includes an API application program interface so that the monitoring mobile terminal can retrieve relevant data through the API application program interface.

[0054] In summary, according to the network video monitoring system of the embodiment of the present invention, a video stream receiving module is set to receive the corresponding multiple video stream data pushed by multiple front-end devices, and the multiple video stream data are transmitted; the video stream real-time preview module is used to send the received multiple video stream data to multiple monitoring mobile terminals, so that the multiple monitoring mobile terminals can preview the multiple video stream data in real time; the video stream playback recording module is used to process the received multiple video stream data to obtain a video file so that the multiple monitoring mobile terminals can play back the video file; the unified storage module is used to back up and store the received video file so that the multiple monitoring mobile terminals can retrieve the backed-up video file; the video stream processing module is used to generate thumbnails, detect motion and identify objects for the multiple video stream data, and store the obtained identification results so that the multiple monitoring mobile terminals can retrieve them; the video stream distribution module is used to transmit the multiple video stream data to the content distribution network through the real-time message transmission protocol so that the multiple monitoring mobile terminals can watch them online; thus, by transmitting, storing and managing the video streams of the monitoring cameras on the video monitoring cloud platform, multi-functional video monitoring is achieved with low cost and high reliability, thereby greatly improving the user experience.

[0055] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0056] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0057] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0058] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0059] It should be noted that in the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claim. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0060] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0061] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0062] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0063] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0064] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0065] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms should not be understood as necessarily referring to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0066] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A network video monitoring system, characterized in that: It includes multiple monitoring mobile terminals, a video monitoring cloud platform and multiple front-end devices, wherein the video monitoring cloud platform is connected to the multiple monitoring mobile terminals and the multiple front-end devices respectively, and the video monitoring cloud platform includes a video stream receiving module, a video stream real-time preview module, a video stream playback and recording module, a unified storage module, a video stream processing module and a video stream distribution module; The video stream receiving module is used to receive the corresponding multiple video stream data pushed by the multiple front-end devices, and transmit the multiple video stream data; The video stream real-time preview module is used to send the received multiple video stream data to the multiple monitoring mobile terminals, so that the multiple monitoring mobile terminals can preview the multiple video stream data in real time; The video stream playback and recording module is used to process the received multiple video stream data to obtain a video file so that the multiple monitoring mobile terminals can play back the video file; The unified storage module is used to back up and store the received video files so that the multiple monitoring mobile terminals can retrieve the backed-up video files; The video stream processing module is used to generate thumbnails, detect motion and identify objects for the multiple video stream data, and store the obtained identification results so that the multiple monitoring mobile terminals can retrieve them; The video stream distribution module is used to transmit the plurality of video stream data to a content distribution network through a real-time message transmission protocol so that the plurality of monitoring mobile terminals can watch the video online; The unified storage module includes a hierarchical storage module, a quota judgment module and a periodic sedimentation module. The hierarchical storage module uses S3 or S3-compatible to perform hierarchical storage on the video files. The quota judgment module is used to limit the available capacity of the hierarchical storage module according to preset conditions, or to limit the total available storage of a certain monitoring mobile terminal according to the configuration, and to provide two optional modes of alarm or overwrite when the limit is full / exceeded. The periodic sedimentation module is used to regularly sediment the video file backup from hot storage to low-frequency / archival access type according to the settings, and to perform archival storage thawing when receiving the backup video file retrieved by the monitoring mobile terminal; the periodic sedimentation module automatically selects extreme speed or normal speed to thaw the file according to the file size after receiving the retrieval request, and performs early thawing when the video playback progress is close to the archive storage file block; Among them, the video stream processing module includes a motion detection module, a thumbnail generation module and an object recognition module, wherein the motion detection module is used to perform frame extraction processing on each of the multiple video stream data and judge the picture motion according to the image through a frame difference method or a background segmentation method to obtain a motion detection result, the thumbnail generation module is used to extract frames for each video stream data according to a fixed interval to generate a thumbnail and write it into storage, and the object recognition module is used to realize object recognition through the Google TensorFlow&DeepStack artificial intelligence suite to obtain a recognition result.

2. The network video monitoring system according to claim 1, characterized in that: The multiple front-end devices are cameras supporting different formats.

3. The network video monitoring system according to claim 2, characterized in that: The video stream receiving module includes an RTMP module, an SRT module and a GB28181 module. After the multiple cameras send their own device numbers, supported video formats, and encoding information to the signaling service to complete device registration, the video stream receiving module returns the RTMP / SRT endpoint address, and for the GB28181 protocol, returns SIP registration information to the corresponding multiple cameras, so that the multiple cameras can push their respective video stream data to the module corresponding to the video stream receiving module.

4. The network video monitoring system according to claim 1, characterized in that: The video stream real-time preview module includes an HLS module, a WebRTC module and a DASH module, wherein the HLS module is used to provide good compatibility for the multiple monitoring mobile terminals, the WebRTC module is used to provide ultra-low latency monitoring real-time viewing, and the DASH module is used to divide each of the multiple video stream data into multiple file segments with multiple bit rates, so that each of the multiple monitoring mobile terminals automatically selects to download and play a corresponding file segment according to current network conditions.

5. The network video monitoring system according to claim 1, characterized in that: The video stream playback and recording module includes a real-time playback module, a historical playback module and a playback export module, wherein the real-time playback module adopts a long-window HLS live broadcast to provide near-line playback, the historical playback module dynamically generates an HLS index file as needed to provide historical playback, and the playback export module provides a packaged FLV format video file for exporting playback.

6. The network video monitoring system according to any one of claims 1 to 5, characterized in that: The monitoring mobile terminal can be any one of a mobile phone, a computer or a tablet.

7. The network video monitoring system according to claim 6, characterized in that: The network video monitoring system also includes a notification and alarm module, which is used to receive notification information generated by each module and push it to the corresponding monitoring mobile terminal according to the contact information and channel configuration corresponding to the notification information.

8. The network video monitoring system according to claim 7, characterized in that: The network video monitoring system also includes an API application program interface so that the monitoring mobile terminal can retrieve relevant data.

Images