Video compression transmission system and method based on multi-level video cascade monitoring

Abstract

The invention discloses a video compression transmission system and method based on multi-level video cascade monitoring, and the system comprises a group headquarters and multi-level enterprise sides, and each level of enterprise side comprises a plurality of current-level enterprise sides. The video compression and transmission system comprises a video decoding module, a background image service module and a plurality of video compression and transmission modules, each enterprise terminal is provided with a video compression and encryption module, and the video decoding module and the background image service module are arranged in the headquarters of a group. According to the invention, through segmentation, compression, encryption and transmission of the monitoring video, the network bandwidth occupation during monitoring video transmission is reduced, the network bandwidth use cost is saved, the video data transmission security is improved, the video time delay caused by multi-level video cascading is reduced, and the video transmission real-time performance is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of video transmission, and more particularly to a video compression transmission system and method based on multi-level video cascade monitoring. Background Art

[0002] With the development of video technology, many group-type central enterprises, state-owned enterprises or large private enterprises have installed surveillance cameras in key areas such as production sites and storage warehouses of subordinate enterprises in order to facilitate the supervision and management of production safety of many subordinate enterprises or the monitoring of important materials. These surveillance videos are uploaded step by step through video cascade technology and sent to sub-groups and group headquarters. Through the surveillance video management platform built by the group headquarters, these surveillance videos are analyzed, processed and inspected according to different business needs.

[0003] However, these large-scale video cascade monitoring systems generally have the following problems: First, the subsidiaries of group enterprises are generally far away from the headquarters and are located in different cities or provinces. In order to ensure the quality and real-time performance of video transmission, these monitoring systems are built using the private network provided by telecommunications companies. The cost of telecommunications private networks is relatively high and the network bandwidth is limited; second, when uploading monitoring videos from numerous lower-level subsidiaries to the group headquarters, there are risks to data security; finally, when the monitoring videos are uploaded from the lower-level enterprises step by step, the video data generally needs to be re-packaged and forwarded when passing through each level of enterprise. Each time the video is forwarded, the delay will increase. The more forwarding levels, the greater the delay. This will cause great problems for scenarios with high real-time requirements.

[0004] Therefore, how to reduce the network bandwidth cost of video cascade transmission and improve the security and real-time transmission of video data is an urgent problem that technicians in this field need to solve. Summary of the invention

[0005] In view of this, the present invention provides a video compression transmission system and method based on multi-level video cascade monitoring, which can reduce the network bandwidth occupancy during monitoring video transmission, save network bandwidth usage fees, improve the security of video data transmission, and reduce the video delay caused by multi-level video cascade, thereby improving the real-time performance of video transmission.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A video compression transmission system based on multi-level video cascade monitoring, the system for realizing multi-level video cascade monitoring includes a group headquarters and multi-level enterprise sides, each level of enterprise side includes a number of enterprise terminals at the same level; the video compression transmission system includes: a video decoding module, a background image service module and a number of video compression transmission modules;

[0008] A video compression and encryption module is configured at each enterprise end at the same level to perform background segmentation, compression and encryption processing on the collected original video compression data to generate background image compression ciphertext and foreground video stream compression ciphertext; the enterprise end saves the foreground video stream compression ciphertext, and the video compression and encryption module transmits the background image compression ciphertext to the background image service module;

[0009] A video decoding module and a background image service module are configured at the group headquarters, and the background image compressed ciphertext is received and stored through the background image service module; the group headquarters generates a video playback request and transmits it to the corresponding enterprise end, and receives the foreground video stream compressed ciphertext playback address returned by the enterprise end; the group headquarters pulls the foreground video stream compressed ciphertext from the corresponding enterprise end according to the foreground video stream compressed ciphertext playback address, and at the same time retrieves the background image compressed ciphertext from the background image service module according to the video playback request; the video decoding module decrypts, decodes and merges the background image compressed ciphertext and the foreground video stream compressed ciphertext to obtain a complete video stream, and the group headquarters plays the complete video frame.

[0010] Preferably, the group headquarters includes a group video surveillance service platform and a group video service gateway; the group video surveillance service platform includes an enterprise monitoring equipment list display module, a video display module and a streaming media receiving module; the group video service gateway includes a group video cascade service module and a streaming media service module; the video decoding module is loaded on the group video surveillance service platform, and the background image service module is loaded on the group video service gateway.

[0011] Preferably, the enterprise end includes an enterprise video source and an enterprise video service gateway; the enterprise video service gateway includes a video access gateway module, a monitoring data storage module and an enterprise video cascade service module, the enterprise video cascade service module is connected to the group video cascade service module and the enterprise video cascade service module of the next-level enterprise end; the video compression and encryption module is loaded on the enterprise video service gateway, and the background image sending module of the video compression and encryption module is connected to the background image receiving module of the background image service module. The compressed ciphertext of the historical background image is stored in the background image service module; the compressed ciphertext of the historical foreground video stream is stored in the monitoring data storage module of the enterprise video service gateway.

[0012] Preferably, the video compression transmission module includes a video intelligent compression module, a monitoring data encryption module and a background image sending module;

[0013] The video intelligent compression module receives the original video compression data collected by the enterprise end and the national standard code of the corresponding monitoring equipment, and performs background segmentation to obtain the background image and foreground YUV video stream, compresses the background image to generate a compressed background image, and compresses the foreground YUV video stream to generate a complete foreground video stream;

[0014] The monitoring data encryption module receives the compressed background image and the complete foreground video stream, and performs encryption processing respectively to obtain the compressed ciphertext of the background image and the compressed ciphertext of the foreground video stream; stores the compressed ciphertext of the foreground video stream in the monitoring video storage module of the enterprise end at the same level and sends it to the enterprise video cascade service module of the enterprise end at the previous level, or sends it to the group headquarters through the enterprise video cascade service module of the enterprise end at the same level according to the video playback request;

[0015] The background image sending module receives the compressed ciphertext of the background image and transmits it to the background image service module; receives the video playback request from the enterprise side, reads the corresponding compressed ciphertext of the background image from the enterprise side and sends it to the group headquarters.

[0016] In the above technical scheme, when the video playback request is for real-time playback, the streaming media receiving module of the group video surveillance service platform obtains the real-time foreground video stream compressed ciphertext from the monitoring data encryption module of the video compression encryption module through the enterprise video cascade service module according to the foreground video stream compressed ciphertext playback address, and the background image receiving module in the background image service module of the group headquarters obtains the updated real-time background image compressed ciphertext and transmits it to the streaming media service module; when the video playback request is for historical playback, the streaming media receiving module of the group video surveillance service platform obtains the historical foreground video stream compressed ciphertext from the monitoring data storage module through the enterprise video cascade service module according to the foreground video stream compressed ciphertext playback address, and the background image storage module in the background image service module of the group headquarters obtains the historical background image compressed ciphertext and transmits it to the streaming media service module.

[0017] Preferably, the video intelligent compression module includes a monitoring decoding module, a monitoring video background acquisition module, a monitoring video foreground acquisition module, a background image management module, a background image compression module and a foreground video compression module;

[0018] The monitoring decoding module receives the original video compression data collected by the enterprise end and the national standard code of the corresponding monitoring equipment, and decodes the original video compression data to obtain the monitoring video;

[0019] The surveillance video foreground acquisition module extracts video streams from the surveillance video in sequence according to the preset time window, takes the first video frame in the video stream as the background image or obtains the most recent background image from the background image management module; uses the image segmentation algorithm to perform background segmentation on each video frame in each video stream, and extracts several foreground YUV video streams;

[0020] The foreground video compression module compresses each foreground YUV video stream, determines whether adjacent foreground YUV video streams are continuous based on the start recording time, and packages multiple continuous compressed foreground YUV video streams into a complete foreground video stream;

[0021] The monitoring video background acquisition module compares each video frame in the monitoring video according to the foreground YUV video stream, selects the video frame with the smallest contrast difference value as the background image and transmits it to the background image management module; the background image includes the A-type background image transmitted according to the network idle rule and the B-type background image transmitted in real time;

[0022] A background image management module receives the background image, saves it, and sends it to the background compression module;

[0023] A background image compression module compresses the background image, generates a compressed background image and sends it to the monitoring data encryption module; compresses the Class A background image, Class B background image and the corresponding recording time and national standard code, generates a compressed background image and sends it to the monitoring data encryption module;

[0024] The monitoring data encryption module encrypts the compressed background image using a preset encryption algorithm, obtains the compressed ciphertext of the background image and sends it to the background image sending module; encrypts the complete foreground video stream using a preset encryption algorithm, obtains the compressed ciphertext of the foreground video stream and sends it to the enterprise video cascade service module and the monitoring data storage module in the enterprise video service gateway at the enterprise end.

[0025] Preferably, the process of extracting the foreground YUV video stream by the monitoring video foreground acquisition module is:

[0026] Step 11: extracting video streams from the surveillance video in sequence according to the preset time window, taking the first video frame in the video stream as a reference background image, or retrieving the most recent background image from the background image management module as a reference background image;

[0027] Step 12: Use an image segmentation algorithm to segment each video frame in the video stream, extract the foreground image and construct a foreground YUV video stream; directly use the image segmentation algorithm to segment the image into a foreground image and a background image, where the segmented background image is a partial image;

[0028] Step 121: Segment one or more initial foreground images from each video frame, mark the maximum range of all initial foreground images in the four directions of up, down, left, and right in the reference background image, and obtain four sets of coordinate positions (a1, a2, b1, b2);

[0029] Step 122: construct a rectangle that completely covers the four sets of coordinate positions, and obtain the coordinates of the four corner points of the rectangle (c1, c2, d1, d2);

[0030] Step 123: extract the image within the rectangular range (c1, c2, d1, d2) as the foreground image, and the foreground images of all video frames constitute a foreground YUV video stream;

[0031] Step 13: Extract the image within the rectangular range (c1, c2, d1, d2) from the reference background image as the motion judgment background;

[0032] Step 14: Calculate the difference between each foreground image and the motion judgment background in the foreground YUV video stream as the foreground change value. When there is at least one set of foreground change values ​​greater than or equal to the set motion threshold Q, it is determined that there is motion change in the foreground YUV video stream, and proceed to step 15; otherwise, there is no motion change in the foreground YUV video stream, and proceed to step 16;

[0033] Step 15: Send the foreground YUV video stream and the corresponding start recording time, the coordinate position relative to the reference background image, and the national standard code to the foreground video compression module, extract the video stream of the next continuous time period for processing, and return to step 12;

[0034] Step 16: Abandon the current foreground YUV video stream, extract the video stream of the next continuous time period for processing, and return to step 12.

[0035] Preferably, the specific process of extracting the background image by the monitoring video background acquisition module is:

[0036] Step 21: retrieve the most recent background image from the background image management module as a reference background image, and remove pixels within a rectangular range (c1, c2, d1, d2) in the reference background image to obtain a removed reference background image;

[0037] Step 22: Eliminate pixels within a rectangular range (c1, c2, d1, d2) in each video frame in the surveillance video to obtain a eliminated video frame;

[0038] Step 23: Calculate the difference between the removed video frame and the removed reference background image corresponding to each video frame as the background change value;

[0039] Step 24: When the background change value is less than the preset background small change threshold, it is determined that the background change of the current video frame relative to the reference background image is small, the current video frame is discarded, and the reference background image is not updated; when the background change value is greater than or equal to the preset background small change threshold, and less than or equal to the preset background large change threshold, the current video frame is marked as a Class A background image, and the Class A background image, recording time and national standard code are sent to the background image management module; when the background change value is greater than the background large change threshold, the current video frame is marked as a Class B background image, and the video frame corresponding to the Class B background image, recording time and national standard code are sent to the background image management module.

[0040] Preferably, the background image management module sends Class A background images and Class B background images; sends the first Class A background image and the corresponding recording time and national standard code to the background image compression module in real time, and sends the remaining Class A background images and the corresponding recording time and national standard code to the background image compression module according to the network idle sending rules configured in the management background of the enterprise video service gateway, and only saves the most recent Class A background image; sends the Class B background image and the corresponding recording time and national standard code to the background image compression module in real time, and only saves the most recent Class B background image.

[0041] Preferably, the complete foreground video stream includes video header data and video frame data, the video header data includes the start recording time, video frame rate per second, video encoding format, the coordinate position (c1, c2, d1, d2) of the corresponding foreground YUV video stream located at the reference background image and the national standard encoding.

[0042] Preferably, when the video intelligent compression module runs for the first time, the background image management module does not store the background image. Before the video intelligent compression module runs for the first time, a background image is collected during non-working hours and saved to the background image management module. The specific process is: the monitoring decoding module preferentially sends a set number of video frames to the monitoring video background acquisition module of the video intelligent compression module, and the monitoring video background acquisition module calculates the difference value between all video frames. When the difference value is less than the difference threshold P configured in the management background of the enterprise video service gateway, the last frame of the video frame is used as a Class A background image, and the Class A background image and the recording time are sent to the background image management module; when all the difference values ​​are greater than or equal to the difference threshold P, the monitoring video background acquisition module continues to receive new video frames and repeatedly calculates the difference value until the difference value is less than the difference threshold P.

[0043] Preferably, the background image service module includes a background image receiving module and a background image storage module; the background image receiving module receives the background image compressed ciphertext transmitted by the background image sending module, and transmits it to the background image storage module for storage, or transmits it to the streaming media service module of the group headquarters according to the video playback request; when the video playback request is real-time playback, the background image receiving module receives the background image compressed ciphertext in real time and transmits it to the streaming media service module, and when the video playback request is historical playback, the background image storage module retrieves the background image compressed ciphertext and transmits it to the streaming media service module.

[0044] Preferably, the video decoding module receives the compressed ciphertext of the foreground video stream transmitted by the group headquarters or the enterprise end, and the compressed ciphertext of the background image transmitted by the background image service module, and performs decryption, decoding and merging processing to obtain a complete video frame; the video decoding module includes a monitoring data decryption module and a video decoding synthesis module;

[0045] The monitoring data decryption module uses a preset decryption algorithm to decrypt the compressed ciphertext of the foreground video stream to obtain the complete foreground video stream, and decrypt the compressed ciphertext of the background image to obtain the compressed background image;

[0046] The video decoding and synthesis module decodes the complete foreground video stream to obtain a foreground YUV video stream; decodes the compressed background image to obtain a background image; merges each foreground video frame in the foreground YUV video stream with the background image, covers the pixels at the corresponding position in the background image with the content in the foreground video frame, and adds the background image with the closest time between the two foreground YUV video streams to obtain a complete video stream, which is then sent to the video display module of the group headquarters for playback.

[0047] Preferably, the process of data processing by each module in the group headquarters includes:

[0048] The enterprise equipment list display module generates and displays a monitoring equipment list based on the video access and video cascade information of all enterprises in the group; generates a video playback request through the selection of the monitoring equipment list and transmits it to the streaming media receiving module; the video playback request includes the playback type and national standard encoding, and the playback type includes real-time playback and historical playback;

[0049] The streaming media receiving module receives the video playback request and sends it to the streaming media service module, and receives the returned foreground video stream compressed ciphertext playback address and background image compressed ciphertext; pulls the corresponding foreground video stream compressed ciphertext from the corresponding enterprise video service gateway according to the foreground video stream compressed ciphertext playback address; transmits the foreground video stream compressed ciphertext and background compressed ciphertext to the monitoring data decryption module of the video decoding module;

[0050] The streaming media service module receives the video playback request transmitted by the streaming media receiving module, sends it to the group video cascade service module, and receives the returned foreground video stream compressed ciphertext playback address and transmits it to the streaming media receiving module; reads the background image compressed ciphertext from the background image storage module according to the playback type of the video playback request, or reads or receives the background image compressed ciphertext from the background image storage module and the background image receiving module at the same time; if it is real-time playback, reads the latest background image compressed ciphertext of the Class A background image from the background image storage module as the current starting background image, and receives the background image compressed ciphertext of the Class A background image and the Class B background image from the background image receiving module; if it is historical playback, reads the background image compressed ciphertext of the Class A background image and the Class B background image of the corresponding time period from the background image storage module;

[0051] The group video cascade service module receives the compressed ciphertext of the foreground video stream; receives the monitoring equipment information of all enterprise ends transmitted by the group video cascade service module, obtains the video access and video cascade information of all enterprises in the group according to the monitoring equipment information, including the IP address, service port, streaming media transmission protocol and national standard encoding of each monitoring video, and transmits the video access and video cascade information of all enterprises in the group to the enterprise equipment list display module; receives the video playback request sent by the streaming media service module, and transmits it to the enterprise video cascade service module in the enterprise video service gateway of the corresponding lower-level enterprise according to the video access and video cascade information of all enterprises in the group, and receives the corresponding foreground video stream compressed ciphertext playback address fed back by the enterprise video cascade service module according to the national standard encoding carried in the video playback request, and sends the foreground video stream compressed ciphertext playback address to the streaming media service module.

[0052] Preferably, the data processing process of each module on the enterprise side includes:

[0053] The video access gateway module is used to receive the original video compression data of the enterprise video source and read the national standard code of the monitoring equipment corresponding to the original video compression data;

[0054] The video compression and encryption module receives the original video compression data and the national standard code, and decodes, segments and compresses the original video compression data to generate background image compression ciphertext and foreground video stream compression ciphertext; sends the background image compression ciphertext to the background image receiving module in the background image service module of the group headquarters, and sends the foreground video stream compression ciphertext to the monitoring data storage module and the enterprise video cascade service module;

[0055] The monitoring data storage module stores the compressed ciphertext of the foreground video stream; receives the reading instructions of the background image sending module and the enterprise video cascade service module, and retrieves the compressed ciphertext of the background image in the corresponding time period;

[0056] The enterprise video cascade service module receives the compressed ciphertext of the foreground video stream sent by the monitoring data encryption module of the video compression encryption module, and sends it to the group video cascade service module; receives the video playback request, and when the video playback request is of the historical playback type, reads the corresponding compressed ciphertext of the foreground video stream from the monitoring data storage module according to the playback time and national standard encoding in the video playback request, and sends it to the group video cascade service module; receives the monitoring equipment information of the local enterprise end and the lower-level enterprise end, generates all video access and video cascade information of the local and lower-level enterprises, including each monitoring The IP address, service port, streaming media transmission protocol, national standard encoding, etc. of the video; report the monitoring equipment information of the local enterprise end and the lower-level enterprise end to the upper-level enterprise end through the video cascade protocol service, until it is transmitted to the group video cascade service module of the group headquarters; perform signaling interaction with the enterprise video cascade service module of the upper-level enterprise end or the lower-level enterprise end through the video cascade protocol service, including handshake negotiation of the video transmission protocol, and upload the streaming media playback address of the compressed ciphertext of the foreground video stream of the enterprise end at this level to the upper-level enterprise end according to the video playback requirements sent by the upper-level enterprise end.

[0057] Preferably, the group video surveillance service platform also includes other business service modules for providing other business services.

[0058] A video compression transmission method based on multi-level video cascade monitoring includes the following steps:

[0059] Step 1: Perform background segmentation, compression and encryption on the surveillance videos collected by the enterprise terminals at all levels to generate background image compression ciphertext and foreground video stream compression ciphertext; the enterprise terminal saves the foreground video stream compression ciphertext and transmits the background image compression ciphertext to the group headquarters;

[0060] Step 2: The group headquarters receives the compressed ciphertext of the background image and saves it;

[0061] Step 3: The group headquarters generates a video playback request and transmits it to the corresponding enterprise end, and receives the compressed ciphertext playback address of the foreground video stream returned by the enterprise end;

[0062] Step 4: The group headquarters pulls the compressed ciphertext of the foreground video stream from the corresponding enterprise end according to the compressed ciphertext playback address of the foreground video stream, and at the same time retrieves the compressed ciphertext of the background image according to the video playback request;

[0063] Step 5: Decrypt, decode and merge the retrieved background image compressed ciphertext and foreground video stream compressed ciphertext to obtain a complete video stream for playback.

[0064] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a video compression transmission system and method based on multi-level video cascade monitoring, which has the following beneficial effects:

[0065] (1) The foreground image and background image of the surveillance video are divided and compressed and transmitted separately. For background images that have a significant impact on business changes, they are compressed and transmitted in real time in picture format. For background images with relatively small content changes, they are compressed and transmitted when the network is idle. Foreground images whose content changes exceed a specified threshold are compressed and transmitted in the form of video streams. Foreground images with basically no content changes do not need to be transmitted. The user terminal at the group headquarters decodes the compressed and encrypted foreground video stream and background image into a complete video frame. This can significantly reduce the telecommunications gateway bandwidth occupied by surveillance video transmission without affecting business, thereby reducing the operating cost of the surveillance system.

[0066] (2) The compressed background image and foreground video stream of the surveillance video are encrypted before transmission. The user terminal first decrypts the received background image and foreground video stream, and then decodes them to synthesize the complete surveillance video, ensuring the security of the enterprise surveillance video data in all transmission links.

[0067] (3) After the foreground video stream is compressed and processed at the enterprise end, it is encapsulated and processed by the streaming media service of the enterprise video service gateway at the same level, and the foreground video playback address is sent to the user terminal where the video decoding module is located through the enterprise video service gateway at the same level, as well as the cascaded upper-level enterprise video service gateway and the group video service gateway. The user terminal receives the foreground video stream directly from the streaming media service of the enterprise video service gateway where the enterprise monitoring video source is located through the foreground video playback address for decoding and playback, thereby avoiding the delay caused by the general enterprise video service gateway re-encapsulating and forwarding the video stream layer by layer, and improving the real-time performance of the user terminal in receiving and playing the enterprise monitoring video data. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0069] Figure 1 The video compression transmission system architecture and data transmission schematic diagram based on multi-level video cascade monitoring provided by the present invention;

[0070] Figure 2 A schematic diagram of the video intelligent compression module flow provided by the present invention. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] The embodiment of the present invention discloses a video compression transmission system based on multi-level video cascade monitoring, such as Figure 1 As shown, the system for realizing multi-level video cascade monitoring includes a group headquarters and multi-level enterprise sides, each level of enterprise side includes several enterprise terminals at the same level, and the video compression transmission system includes: a video decoding module, a background image service module and several video compression transmission modules; a video compression encryption module is configured at each enterprise terminal of each level of enterprise side, which is used to perform background segmentation, compression and encryption processing on the collected original video compression data, and generate background image compression ciphertext and foreground image compression ciphertext; the enterprise terminal saves the foreground video stream compression ciphertext, and the video compression encryption module transmits the background image compression ciphertext to the background image service module; the video decoding module is configured at the group headquarters The group headquarters generates a video playback request and transmits it to the corresponding enterprise end, and receives the foreground video stream compressed ciphertext playback address returned by the enterprise end; the group headquarters pulls the foreground video stream compressed ciphertext from the corresponding enterprise end according to the foreground video stream compressed ciphertext playback address, and at the same time retrieves the background image compressed ciphertext from the background image service module according to the video playback request; the video decoding module decrypts, decodes and merges the background image compressed ciphertext and the foreground video stream compressed ciphertext to obtain a complete video stream, and the group headquarters plays the complete video stream.

[0073] The group headquarters includes a group video surveillance service platform and a group video service gateway; the group video surveillance service platform includes an enterprise monitoring equipment list display module, a video display module and a streaming media receiving module; the group video service gateway includes a group video cascade service module and a streaming media service module; the video decoding module is loaded on the group video surveillance service platform, and the background image service module is loaded on the group video service gateway.

[0074] The enterprise end includes the enterprise video source and the enterprise video service gateway; the enterprise video service gateway includes the video access gateway module, the monitoring data storage module and the enterprise video cascade service module, the enterprise video cascade service module is connected to the group video cascade service module and the enterprise video cascade service module of the next level enterprise end; the video compression and encryption module is loaded on the enterprise video service gateway, and the background image sending module of the video compression and encryption module is connected to the background image receiving module of the background image service module. The compressed ciphertext of the historical background image is stored in the background image service module; the compressed ciphertext of the historical foreground video stream is stored in the monitoring data storage module of the enterprise video service gateway.

[0075] The video compression transmission module includes a video intelligent compression module, a monitoring data encryption module and a background image sending module; the video intelligent compression module includes a monitoring decoding module, a monitoring video background acquisition module, a monitoring video foreground acquisition module, a background image management module, a background image compression module and a foreground video compression module; the background image service module includes a background image receiving module and a background image storage module; the video decoding module includes a monitoring data decryption module and a video decoding synthesis module.

[0076] On the other hand, in a specific embodiment, Figure 1 As shown, the group headquarters includes a group video service gateway and a group video surveillance service platform. The group video service gateway is used to receive surveillance video data uploaded by all levels of enterprises under the group headquarters, and send the surveillance video data to the group video surveillance service platform; the group video surveillance service platform is used to display a list of all enterprise surveillance devices connected to the group, receive enterprise surveillance video data according to user requests, decrypt the video data, decode the foreground video and background image respectively, and then merge the foreground video and background image into a complete surveillance video for display and playback, while providing other business service modules required by the group.

[0077] The enterprise side includes enterprise video sources and enterprise video service gateways. The enterprise video sources are used to provide monitoring video data, including various monitoring devices or monitoring platforms, such as IPC cameras, NVR devices, and HaiDaYu's video monitoring platform. The enterprise video service gateway is used to receive monitoring video data from the enterprise video source, divide the monitoring video data into background images and foreground videos through analysis and calculation, and compress them separately. The compressed foreground video data is encrypted and saved locally on the enterprise video service gateway, and the compressed background image is encrypted and uploaded to the group video service gateway.

[0078] The enterprise video service gateway is ultimately connected to the group headquarters through video cascading. The group headquarters and subordinate enterprises are divided into multiple levels, such as the group headquarters, second-level enterprises, third-level enterprises, and so on. The lowest-level enterprise video service gateway is first connected to the enterprise video service gateway of the upper-level enterprise, and then finally connected to the group video service gateway through a layer-by-layer cascading method, realizing the video cascading of all lower-level enterprises under the group. For example, when the lowest-level enterprise is a third-level enterprise, the enterprise video service gateways of all third-level enterprises are first connected to the enterprise video service gateway of the second-level enterprise directly above, and then the video service gateways of all second-level enterprises are connected to the group video service gateway.

[0079] On the other hand, in a specific embodiment, Figure 1 As shown, the enterprise video service gateway includes a video access gateway module, a monitoring data storage module and an enterprise video cascading service module.

[0080] The video access gateway module receives video data from the enterprise video source, extracts the original video compression data from the network transmission layer, reads the national standard code assigned to the surveillance video in the configuration information for identifying the video, and finally transmits the original video compression data and the national standard code to the video intelligent compression module.

[0081] like Figure 2 As shown, the video intelligent compression module includes a monitoring decoding module, a monitoring video background acquisition module, a background image management module, a background image compression module, a monitoring video foreground acquisition module, and a foreground video compression module. The monitoring decoding module decodes the original video compression data received from the video access gateway module, and outputs the video frames in YUV format, the recording time of each frame, the national standard encoding of the video, and other data to the subsequent processing module.

[0082] The surveillance video foreground acquisition module, according to the configuration parameters of the management background of the enterprise video service gateway, receives the specified number of time-continuous video frame data output by the surveillance decoding module as a video processing unit, such as a video stream of 1800 frames of about 1 minute, and then reads the time-recent background image of the surveillance video from the background image management module as the reference background image. If the background image management module has not yet generated the first background image of the surveillance video, the first frame image received is used as the reference background image. The surveillance video foreground acquisition module uses a certain foreground and background separation algorithm, such as an algorithm based on image segmentation, a deep learning algorithm, a graph model algorithm, a motion analysis algorithm, etc., to process each video frame in the video stream and extract the foreground image of each video frame. Generally, there are one or more foreground images in each video frame, and the size and position of the foreground image extracted from each frame of the video stream are generally different. Mark the maximum range of the upper, lower, left and right positions of all foreground images in the reference background image to obtain four coordinate positions: the upper left corner is a1, the upper right corner is a2, the lower left corner is b1, and the lower right corner is b2. The figure formed by the combination of these four coordinate positions is generally an irregular figure. The rectangle with coordinates c1, c2, d1, d2 is analyzed and calculated. This rectangle is a minimum rectangle that can completely cover the above irregular figure, and the four sides of the rectangle are parallel to the four sides of the background reference image. Reprocess each frame of the video unit and extract the foreground image with coordinates c1, c2, d1, d2. In this way, a rectangular foreground YUV video stream with the same shape in each frame can be formed. The extracted foreground YUV video stream can cover all the foreground images with changes in the content extracted previously. Process the reference background image and extract the background image with coordinates c1, c2, d1, d2. Compare and calculate each frame of the foreground video stream with the processed reference background image. When the difference value is greater than the threshold set in the enterprise video service gateway management background, it indicates that there is motion change in the foreground YUV video stream. The extracted foreground YUV video stream is sent to the foreground video compression module for processing. At the same time, the data sent includes the start recording time of the video segment, the coordinate position of the video image in the reference background image (c1, c2, d1, d2) and the national standard encoding of the surveillance video of the image segment. Then the surveillance video foreground acquisition module continues to receive and process subsequent video unit frames with continuous processing time. When the difference value is less than the set threshold, it indicates that there is no motion change in the foreground video stream, and the processing of the foreground YUV video stream is abandoned, and the subsequent video streams with continuous processing time are continued to be received.

[0083] The foreground video compression module receives the foreground YUV video stream from the surveillance video foreground acquisition module for video compression. The video compression algorithm used can be the current mainstream video compression algorithm such as H.264 and H.265. The compressed video data is packaged into a foreground video stream. The foreground video compression module checks the start recording time of all received foreground YUV video streams. If it is found that the newly received foreground video stream is discontinuous with the previously received foreground video stream, the newly received foreground video stream is repackaged into a complete foreground video stream. Otherwise, the newly received foreground video stream is connected to the previous foreground video stream to form a complete foreground video stream. The complete foreground video stream contains the video header data at the beginning and each frame of video frame data thereafter. The video header data provides video encoding parameters, including the video start recording time, the video frame rate per second, the video encoding format, the coordinate position of the foreground YUV video stream in the background image (c1, c2, d1, d2), and the national standard encoding of the video.

[0084] The monitoring video background acquisition module obtains the most recent background image from the background image management module as the reference background image, obtains the coordinate position (c1, c2, d1, d2) of the complete foreground video stream from the monitoring video foreground acquisition module, removes the pixels at the corresponding position of the foreground image from the reference background image, and then processes each video frame (YUV video frame data) in the monitoring video received from the video decoding module, and similarly removes the pixels at the corresponding position of the foreground image, and analyzes and compares each processed video frame with the reference background image with the foreground image removed, calculates the difference between each processed video frame and the reference background image with the foreground image removed, and compares this difference with the two thresholds, namely the background small change threshold A1 and the background large change threshold B1, which are pre-set in the management backend of the enterprise video service gateway. When the difference value is less than A1, it means that the frame image has changed very little from the current reference background image, which does not affect the business needs of the group headquarters. There is no need to update the reference background image and the current video frame is discarded; when the difference value is greater than the background small change threshold A1 and less than the background large change threshold B1, it means that there are some changes between the frame image and the current reference background image, but it has no impact on the business needs of the group headquarters. The original image of the processed video frame (i.e., the corresponding video frame in the original surveillance video), recording time and national standard code are sent to the background image management module, and the video frame is marked as a Class A background image, which means that this type of background image can be sent when the network is idle; when the difference value is greater than B1, it means that the frame image has changed significantly from the current reference background image, which has affected the business needs of the group headquarters. The original image of the processed video frame, recording time and national standard code are sent to the background image management module, and the frame image is marked as a Class B background image, which means that this type of background image needs to be sent immediately. When the video intelligent compression module runs for the first time, the background image management module does not save any background images. The monitoring video background acquisition module needs to collect the first background image first. The specific design is: according to the configuration parameters of the management background of the enterprise video service gateway, the monitoring video background acquisition module continuously receives a certain number of video frames sent by the video decoding module during the non-working hours of the enterprise, and calculates the difference between these video frames. When the difference value is less than the threshold P in the management background configuration parameters, the last frame of the image is used as the latest Class A background image and sent to the background image management module, and the recording time of the frame of background image is sent at the same time; if the calculated pixel difference between these video frames is greater than the specified threshold P, continue to receive new video frames and calculate the difference value until it is found that the difference value is less than the specified threshold P.

[0085] The background image management module receives Class A background images and Class B background images from the monitoring video background acquisition module. For Class A background images, they are saved first. If it is the first time that a Class A background image is received, it is immediately sent to the background image compression module. For other received Class A background images, according to the configuration information of the management backend of the enterprise video service gateway, during the specified network idle time period, all saved Class A background images are sent to the background image compression module, and then only the most recent Class A image is saved, and all other saved Class A background images are deleted. For Class B background images, they are forwarded to the background image compression module in real time, and only the most recent Class B background image is saved. The data sent by the background image management module to the background image compression module includes video frames (YUV image frame data), recording time and national standard coding. The background image management module also selects the most recent one from all saved Class A images and Class B images as the current reference background image and sends it to the monitoring video background acquisition module and the monitoring video foreground acquisition module.

[0086] The background image compression module receives the Class A background image and Class B background image sent from the background image management module, compresses the received images, for example, compresses them into JPEG images, and then sends the compressed image data, recording time and national standard code to the monitoring data encryption module.

[0087] The monitoring data encryption module processes the compressed background image and the complete foreground video stream received from the video intelligent compression module separately: the compressed background image containing Class A background image and Class B background image is encrypted, and the compressed ciphertext of the background image obtained after processing is sent to the background image sending module; the video frame data in the complete foreground video stream is encrypted, and the compressed ciphertext of the foreground video stream obtained after processing is sent to the enterprise video cascade service module and the monitoring data storage module.

[0088] The monitoring data storage module stores the foreground video stream data received from the monitoring data encryption module in the local enterprise video service gateway. The stored data also includes the recording time and national standard code. When the enterprise video cascade service module receives the video playback instruction from the upper system, the enterprise video cascade service module will read the corresponding foreground video stream compression ciphertext from the monitoring data storage module according to the playback time requirements and national standard code requirements.

[0089] The background image sending module sends the background image received from the monitoring data encryption module to the background image receiving module in the background image service module of the group video service gateway through the MQTT protocol according to the address information configured in the management backend of the enterprise video service gateway.

[0090] The enterprise video cascade service module mainly includes two functions: video cascade protocol service and streaming media service. The video cascade protocol service mainly includes two parts. The first is to receive the monitoring device registration information of the foreground video stream of the enterprise video source at this level, and at the same time receive the monitoring device registration information of the lower-level enterprise video cascade service module, and generate and save all video access and video cascade information of the enterprise at this level and the lower-level enterprise according to the monitoring device registration information, including all monitoring video information accessed by the enterprise video service gateway at all levels: each channel of monitoring video information includes IP address, service port, streaming media transmission protocol, monitoring video national standard encoding, etc. The enterprise video cascade service module also transmits the registration information of all accessed monitoring devices to the upper level through the video cascade protocol service. Through the layer-by-layer processing of the enterprise video cascade service module device registration, the group video service gateway will obtain the cascade relationship of all enterprise video accesses at all levels; the second is to perform signaling interaction with the upper or lower enterprise video cascade service module according to the video cascade protocol service, including handshake negotiation of the video transmission protocol, uploading the streaming media playback address of the compressed ciphertext of the foreground video stream of the enterprise video source at this level according to the video playback requirements transmitted by the upper level, and other signaling interactions. According to the service requirements of the upper-level system, the streaming media service compresses the ciphertext of the real-time foreground video stream at this level, or the compressed ciphertext of the foreground video stream read from the monitoring data storage module for playback, and pushes it directly to the group monitoring video service platform through the UDP protocol for decoding and playback.

[0091] On the other hand, in a specific embodiment, Figure 1 As shown, the group video service gateway includes a streaming media service module and a group video cascade service module. The background image service module is connected to the streaming media service module. The background image service module includes a background image storage module and a background image receiving module.

[0092] The group video cascade service module is connected to the enterprise video cascade service modules of all secondary enterprises, receives the monitoring device registration reported by the enterprise video cascade service module, generates and saves the video access and video cascade information of all enterprises in the group, including all monitoring video information accessed by the video service gateways of all levels of enterprises. Each monitoring video information includes IP address, service port, streaming media transmission protocol, national standard encoding of monitoring video, etc. The group video cascade service module also sends the generated video access and video cascade information of all enterprises in the group to the group video monitoring service platform. In addition, the group video cascade service module receives the video playback request of the streaming media service module, forwards the playback request to the enterprise video cascade service module, receives the foreground playback address and streaming media transmission protocol of the monitoring video reported by the lower-level enterprise video cascade gateway, and forwards it to the streaming media service module and the streaming media receiving module. The streaming media receiving module pulls the foreground video stream from the enterprise video cascade service module that provides the streaming media service address according to the playback address and streaming media transmission protocol, and sends the video playback control signaling such as play, stop, fast forward, etc. sent by the user through the group video monitoring service platform to the enterprise video cascade service module in the enterprise video service gateway.

[0093] The background image receiving module receives the background image compression ciphertext of each surveillance video sent by the background image sending module in the background compression encryption module of all levels of enterprise video service gateways through the MQTT protocol. The received information includes background image compression data, background image type, background image recording time, national standard code corresponding to the background image, etc. The background image receiving module sends the received background image compression ciphertext to the streaming media service module for real-time video playback scenarios, and sends it to the background image storage module for storage for playback video playback scenarios.

[0094] The streaming media service module receives the video playback request from the group video surveillance service platform. The request parameters include the national standard code of the surveillance video and the playback type (real-time playback or historical playback). The streaming media service module sends the video playback request to the group video cascade service module. The group video service module finds the corresponding secondary enterprise's enterprise video service gateway based on the stored video access and video cascade information of all enterprises in the group, and sends the video playback request to the corresponding enterprise video cascade service module. The enterprise video cascade service module feedbacks the corresponding foreground video stream compression ciphertext playback address based on the national standard code in the video playback request. If it is a video collected by the enterprise video source of the same level enterprise, the real-time foreground video stream compression ciphertext playback address or the historical foreground video stream compression ciphertext playback address of the corresponding surveillance video is obtained according to the playback type and reported. If it is a lower-level enterprise video source, the playback request is forwarded to the lower-level enterprise video service module again, and finally the obtained foreground video stream compression ciphertext playback address is reported to the group video cascade service module and sent to the streaming media service module. The streaming media service module returns the obtained video playback address to the group video surveillance service platform. When the streaming media service module receives a video playback request from the group video surveillance service platform, if it is real-time playback, it reads the latest Class A background image from the background image storage module as the starting background image of the current real-time video playback, and receives Class A background images and Class B background images from the background image receiving module; if it is historical playback, it reads Class A background images and Class B background images of the corresponding time period from the background image storage module.

[0095] The group video surveillance service platform includes an enterprise monitoring equipment list display module, a video decoding module, a video display module, a streaming media receiving module and other business service modules; the video decoding module includes a monitoring data decryption module and a video decoding synthesis module.

[0096] The enterprise monitoring device list display module receives the video access and video cascade information of all enterprises in the group sent by the group video cascade service module to generate a monitoring device list and display it to the user. The user can click to view a certain monitoring video according to his or her needs and generate a video playback request. According to the user's video playback request, the enterprise monitoring device list display module sends the national standard encoding and request type (real-time playback or historical playback) of the video to the streaming media receiving module in the video decoding module.

[0097] The streaming media receiving module receives the video playback request from the enterprise monitoring device list display module, sends the video playback request to the streaming media service module of the group video service gateway, and obtains the compressed ciphertext playback address of the foreground video stream of the corresponding video stream. According to the playback address, the streaming media receiving module pulls the monitoring foreground video stream from the corresponding enterprise video service gateway, and receives the background image compressed ciphertext from the streaming media service module. Depending on the request for real-time playback or historical playback, the corresponding foreground video stream compressed ciphertext and background image compressed ciphertext for real-time playback or historical playback are received. The streaming media receiving module sends the received foreground video stream compressed ciphertext and background image compressed ciphertext to the monitoring data decryption module. The monitoring data decryption module decrypts the received data according to the pre-set decryption algorithm, and outputs the decrypted complete foreground video stream and compressed background image in plain text to the video decoding and synthesis module.

[0098] The video decoding and synthesis module first decodes the received compressed background image (JPEG file) to form a background image of the surveillance video in the YUV format, and then decodes the received complete foreground video stream to form a foreground YUV video stream in the YUV format. Finally, each foreground video frame in the foreground YUV video stream is processed: a background image is copied, and according to the position information of the foreground video frame, the content in the foreground video frame covers the pixels at the corresponding position of the background image to form a complete video frame after the foreground and background are merged. From the screening process of the foreground YUV video stream, it can be seen that if the surveillance video content between two adjacent foreground YUV video streams does not change, the streaming media receiving module will not receive the relevant foreground YUV video stream from the enterprise video service gateway. At this time, the video decoding and synthesis module only needs to send the background image of the corresponding time period (the background image with the closest time can be selected) to the video display module for playback until a new foreground YUV video stream is received. Depending on the actual situation, the video decoding and synthesis module will occasionally receive a new compressed background image with recording time information. Each time a new compressed background image is received, a new background image is decoded and synthesized with the foreground video frame at the corresponding time and thereafter according to the recording time of the new background image. After the new background image starts to be synthesized with the foreground video frame, the old background image is replaced. Before receiving a new background image, the foreground video frame needs to be synthesized with the current background image. The video decoding and synthesis module sends the final synthesized complete video stream to the video display module for video playback and display.

[0099] Other business service modules are used to support other businesses of the group video surveillance service platform, which are not within the scope of the present invention and will not be described further.

[0100] On the other hand, in a specific embodiment, the video compression transmission system compresses and transmits the background and foreground of the surveillance video accessed by the enterprise respectively, including the following steps:

[0101] S11: The user first logs in to the management backend of the enterprise video service gateway to perform configuration work: configure the enterprise surveillance video access information of the enterprise video service gateway at the same level, including IP address, service port, and video transmission protocol; configure the national standard encoding of each surveillance video; configure the number of video frames of the video processing unit; configure the non-working time of the enterprise; configure the foreground motion change detection threshold Q; configure the background detection threshold: the first background image acquisition detection threshold P, the background small change threshold A1, and the background large change threshold B1; configure the background image receiving service module address of the group video service gateway; configure the IP address, service port and other information of the upper-level enterprise video service gateway to be accessed; configure the IP address, service port and other information of all the lower-level enterprise video service gateways to be accessed;

[0102] S12: The enterprise-side video access gateway module receives the video data of the enterprise video source, extracts the original video compression data from the network transmission layer, reads the national standard code assigned to the monitoring video in the configuration information, and transmits the original video compression data and the national standard code to the video intelligent compression module;

[0103] S13: The monitoring decoding module of the video intelligent compression module decodes the original video compression data received from the video access gateway module, and outputs the YUV video frame in YUV format, the recording time of each frame, the national standard encoding of the video, and other data to the subsequent processing module;

[0104] S14: The monitoring video background acquisition module acquires the background image of the first YUV video frame: according to the management background configuration parameters of the enterprise video service gateway, during the non-working hours of the enterprise, continuously receive the specified number of YUV video frames sent by the video decoding module, calculate the difference value between these YUV video frames, and when the difference value is less than the threshold value Q in the configuration parameters, use the last frame of the YUV video frame as the current Class A background image and send it to the background image management module, and send the background image recording time to the background image management module; if the calculated difference value between these YUV video frames is greater than the specified threshold value Q, continue to receive new YUV video frames and calculate the difference value until a YUV video frame with a difference value less than the specified threshold value Q is found, obtain the first Class A background image and send it to the background image management module, and send the background image recording time to the background image management module;

[0105] S15: The monitoring video foreground acquisition module receives the number of time-continuous video frames specified by the configuration parameters from the video decoding module, and reads the latest background image of the monitoring video from the background image management module; uses a certain foreground and background separation algorithm, such as an algorithm based on image segmentation, a deep learning algorithm, a graph model algorithm, a motion analysis algorithm, etc., to process the received video frames and extract the foreground image of each video frame; marks the maximum range of the upper, lower, left and right positions of all foreground images in the background reference image, and obtains four coordinate positions: the upper left corner is a1, the upper right corner is a2, the lower left corner is b1, and the lower right corner is b2. These four coordinates are generally irregular quadrilaterals; analyzes and calculates a rectangle with coordinates c1, c2, d1, and d2, which is a rectangle that can just completely cover the above The minimum rectangle of an irregular figure, and the four sides of the rectangle are parallel to the four sides of the background reference image; reprocess each frame of the video unit, extract the foreground image with coordinate positions c1, c2, d1, d2, and obtain a rectangular foreground YUV video stream with the same shape in each frame; process the background image, extract the background image with coordinate positions c1, c2, d1, d2 as the processed background image; calculate and compare the foreground image of each frame of the foreground YUV video stream with the processed background image, and when the difference value is greater than the specified threshold Q, send the extracted foreground YUV video stream to the foreground video compression module, and the data sent at the same time include the start recording time of the video, the coordinate position of the image in the background image (c1, c2, d1, d2) and the national standard code of the image;

[0106] S16: The foreground video compression module receives the foreground YUV video stream from the monitoring video foreground acquisition module for video compression, packages the compressed foreground video data into a foreground video stream, and packages the subsequent time-continuous foreground video streams into a complete foreground video stream; the complete foreground video stream includes the video header data at the beginning and each subsequent foreground video stream, and the video header data provides video encoding parameters, including video recording time, video encoding format, the coordinate position of the video segment in the background image (c1, c2, d1, d2) and the national standard encoding of the video segment; when the subsequent foreground YUV video stream is discontinuous in time, it is repackaged to form a new complete foreground video stream;

[0107] S17: The monitoring video background acquisition module detects background changes in real time, obtains the most recent background image from the background image management module as a reference background image, obtains the coordinate position (c1, c2, d1, d2) of the foreground YUV video stream from the monitoring video foreground acquisition module, removes the pixels at the corresponding position of the image in the foreground YUV video stream from the reference background image, and also removes the pixels at the corresponding position of the foreground image from each YUV video frame received from the video decoding module, analyzes and compares each processed YUV video frame with the reference background image from which the foreground is removed, calculates the difference value, and compares the difference value with the two values ​​A1 and B1 pre-set in the management backend of the enterprise video service gateway. Thresholds are compared; when the difference value is less than A1, the YUV video frame is not processed; when the difference value is greater than A1 and less than B1, the original image, recording time and national standard code of the processed YUV video frame are sent to the background image management module, and the background image corresponding to the YUV video frame is marked as a type A background image, which means that this type of background image can be sent when the network is idle; when the difference value is greater than B1, the original image, recording time and national standard code of the processed YUV video frame are sent to the background image management module, and the background image corresponding to the YUV video frame is marked as a type B background image, which means that this type of background image needs to be sent immediately;

[0108] S18: The background image management module receives the A-type background image and the B-type background image from the monitoring video background acquisition module; the A-type background image is first saved, and if it is the first time that the A-type background image is received, it is immediately sent to the background compression module; the other A-type background images are sent to the background image compression module according to the configuration information of the management backend of the enterprise video service gateway during the specified network idle time period, and then only the most recent A-type background image is saved, and all other saved A-type background images are deleted; for the B-type background image, it is forwarded to the background image compression module in real time, and only the most recent B-type background image is saved; the data sent by the background image management module to the background image compression module includes the background image (i.e., the YUV video frame of the monitoring video), the recording time and the national standard code;

[0109] S19: A background image compression module receives the class A background image and the class B background image sent from the background image management module, compresses the received background image, for example, compresses it into a JPEG image, and then sends the compressed background image, background image type, recording time and national standard coding information to the monitoring data encryption module;

[0110] 110: The monitoring data encryption module processes the compressed background image and the complete foreground video stream received from the video intelligent compression module respectively: encrypts the JPEG background image and sends the processed background image compressed ciphertext to the background image sending module; encrypts the compressed foreground video data in the complete foreground video stream and sends the processed foreground video stream compressed ciphertext to the enterprise video cascade service module and the monitoring data storage module; when the foreground video stream of the monitoring video is sent to the enterprise video cascade service module for the first time, the monitoring video device registration information is sent first, including the IP address, service address, national standard code and other information of the monitoring video;

[0111] S111: The monitoring data storage module stores the data received from the monitoring data encryption module in the local enterprise video service gateway, and the stored foreground video data includes the recording time and the national standard code to support subsequent video playback;

[0112] S112: The background image sending module compresses the ciphertext of the background image received from the monitoring data encryption module and sends it to the background image receiving service module of the group video service gateway through the MQTT protocol according to the configuration address information of the enterprise video service gateway, or sends it to the enterprise video cascading service module of the upper-level enterprise end through the MQTT protocol according to the configuration address information of the enterprise video service gateway;

[0113] S113: When the enterprise video cascade service module receives the monitoring device registration request sent by the monitoring data encryption module, it verifies it with the video access configuration information saved in the management background. If it is consistent, it is saved locally, and the monitoring device registration request is forwarded to the enterprise video cascade service module in the enterprise video service gateway of the upper enterprise end; when receiving the access monitoring device registration request of the enterprise video service gateway of the lower enterprise end, it is also verified and saved and forwarded to the upper enterprise end, and finally sent to the group video cascade service module of the group video service gateway after forwarding layer by layer; the group video cascade service module receives the device registration information reported by all secondary enterprise video service gateways, and summarizes it to form the video access and cascade information of all levels of enterprises in the whole group;

[0114] S114: The enterprise video cascade service module receives the compressed ciphertext of the foreground video stream sent by the monitoring data encryption module, and encapsulates and processes it into streaming video data through the enterprise streaming service module; when the enterprise video service gateway at the upper-level enterprise end requests to play the streaming video data, it sends the compressed ciphertext playback address of the foreground video stream, and the streaming media receiving module of the group video monitoring service platform finally receives the address. The streaming media receiving module directly connects to the enterprise video cascade service module to send a streaming media service request based on the address information, and the enterprise video cascade service module sends the compressed ciphertext of the foreground video stream to the streaming media receiving module, which is decoded and played through the subsequent service module.

[0115] On the other hand, in a specific embodiment, the video compression transmission system receives monitoring data according to the video playback request of the user, and decodes and synthesizes the background and foreground of the monitoring video, including the following steps:

[0116] S21: The enterprise monitoring device list display module of the group video monitoring service platform receives the video access and video cascading information of all enterprises in the group sent by the group video cascading service module, and displays the list of all enterprise monitoring devices to the user;

[0117] S22: The user logs in to the group video surveillance service platform and can click to view a certain surveillance video according to his needs. When the user clicks, a video playback request is generated; the enterprise surveillance device list display module returns the national standard encoding and playback type (real-time playback or historical playback) of the video according to the user's video playback request, adds it to the video playback request, and sends it to the streaming media receiving module in the video display module;

[0118] S23: The streaming media receiving module receives the video playing request from the enterprise monitoring device list display module and sends it to the streaming media service module of the group video service gateway;

[0119] S24: The streaming media service module receives a video playback request from the group video surveillance service platform, and sends the video playback request to the group video cascading service module;

[0120] S25: The group video cascade service module finds the corresponding secondary enterprise end enterprise video service gateway according to the stored video access and video cascade information of all enterprises in the group, and sends the video play request to the corresponding enterprise video cascade service module;

[0121] S26: The enterprise video cascade service module pushes the compressed ciphertext playback address of the foreground video stream to the group video cascade service module of the group video service gateway in the group headquarters according to the national standard code in the video playback request;

[0122] If the national standard code shows that it is the video source of the enterprise at this level, the real-time foreground video stream compression ciphertext playback address or the historical foreground video stream compression ciphertext playback address of the corresponding monitoring video is obtained according to the playback type and reported to the group video cascade service module; if the national standard code shows that it is the video source of the lower-level enterprise, the video playback request is forwarded again to the enterprise video cascade service module of the lower-level enterprise video service gateway, and the judgment is continued according to the national standard code, and finally the corresponding video playback address is obtained;

[0123] S27: Finally, through the streaming media service module of the group video service gateway and the streaming media receiving module of the group video monitoring service platform, the foreground video real-time playback address or the historical video playback address of the monitoring video requested to be played by the user is obtained;

[0124] S28: The streaming media receiving module receives the compressed ciphertext of the foreground video stream from the corresponding enterprise video service gateway according to the received compressed ciphertext playback address of the foreground video stream, and sends the received data to the monitoring data decryption module of the video decoding module;

[0125] The streaming media receiving module sends a background image request to the streaming media service module of the group video service gateway to apply for receiving the compressed ciphertext of the background image of the corresponding monitoring video according to the national standard code and the playback type in the video playback request; when the streaming media service module receives the background image request, it retrieves the compressed ciphertext of the background image according to the playback type of the video playback request. If it is real-time playback, it is necessary to read the compressed ciphertext of the background image corresponding to the latest Class A background image from the background image storage module as the starting background image of the current real-time video playback, and at the same time receive the compressed ciphertext of the background image corresponding to the Class A background image and the Class B background image from the background image receiving module in real time; if it is historical playback, the compressed ciphertext of the background image corresponding to the Class A background image and the Class B background image of the corresponding historical time period is read from the background image storage module; the streaming media receiving module sends the received compressed ciphertext of the background image to the monitoring data decryption module;

[0126] S29: The monitoring data decryption module decrypts the received data according to a preset decryption algorithm, and outputs the decrypted complete foreground video stream and compressed background image to the video decoding and synthesis module;

[0127] S20: Video decoding and synthesis module, decodes the received compressed background image (JPEG file) to form a background image of the surveillance video in YUV format; decodes the received complete foreground video stream to form a foreground YUV video stream of the surveillance video in YUV format; processes each frame of the foreground image in the foreground YUV video stream, copies a background image for each frame of the foreground image, and according to the position information of each frame of the foreground image, covers the pixels at the corresponding position of the background image with the foreground image content to form a video stream after the foreground and background are merged, and sends it to the video display module;

[0128] If the content of the surveillance video does not change during the current time period, the streaming media receiving module will not receive the compressed ciphertext of the foreground video stream from the enterprise video service gateway, and the video decoding and synthesis module will add the background image of the current surveillance video between the two merged video streams to obtain a complete video stream and send it to the video display module;

[0129] When the video decoding and synthesis module receives a new background image, it synthesizes it with the foreground image and subsequent foreground images in the foreground YUV video stream with the same recording time according to the background image recording time, until a new background image is received and then replaced;

[0130] S221: The video display module displays and plays the complete video stream provided by the video decoding and synthesis module.

[0131] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0132] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A video compression transmission system based on multi-level video cascade monitoring, the system for realizing multi-level video cascade monitoring includes a group headquarters and multi-level enterprise sides, each level of enterprise side includes a number of enterprise terminals at the same level, characterized in that: The video compression transmission system includes: a video decoding module, a background image service module and several video compression transmission modules; A video compression and encryption module is configured at each enterprise end of each level of enterprise side, which is used to perform background segmentation, compression and encryption processing on the collected original video compression data to generate background image compression ciphertext and foreground video stream compression ciphertext; the enterprise end saves the foreground video stream compression ciphertext, and the video compression and encryption module transmits the background image compression ciphertext to the background image service module; A video decoding module and a background image service module are configured at the group headquarters, and the background image compressed ciphertext is received and stored through the background image service module; the group headquarters generates a video playback request and transmits it to the corresponding enterprise end, and receives the foreground video stream compressed ciphertext playback address returned by the enterprise end; the group headquarters pulls the foreground video stream compressed ciphertext from the corresponding enterprise end according to the foreground video stream compressed ciphertext playback address, and at the same time retrieves the background image compressed ciphertext from the background image service module according to the video playback request; the video decoding module decrypts, decodes and merges the background image compressed ciphertext and the foreground video stream compressed ciphertext to obtain a complete video stream, and the group headquarters plays the complete video frame.

2. A video compression transmission system based on multi-level video cascade monitoring according to claim 1, characterized in that: The video compression transmission module includes a video intelligent compression module, a monitoring data encryption module and a background image sending module; The video intelligent compression module receives the original video compression data collected by the enterprise end and the national standard code of the corresponding monitoring equipment, and performs background segmentation to obtain the background image and foreground YUV video stream, compresses the background image to generate a compressed background image, and compresses the foreground YUV video stream to generate a complete foreground video stream; The monitoring data encryption module receives the compressed background image and the complete foreground video stream, and performs encryption processing respectively to obtain the compressed ciphertext of the background image and the compressed ciphertext of the foreground video stream; The compressed ciphertext of the foreground video stream is stored in the enterprise end of the same level and sent to the enterprise end of the previous level, or sent to the group headquarters through the enterprise end of the same level according to the video playback request; The background image sending module receives the compressed ciphertext of the background image and transmits it to the background image service module; receives the video playback request from the enterprise side, reads the corresponding compressed ciphertext of the background image from the enterprise side and sends it to the group headquarters.

3. The video compression transmission system based on multi-level video cascade monitoring according to claim 2 is characterized in that: The video intelligent compression module includes a monitoring decoding module, a monitoring video background acquisition module, a monitoring video foreground acquisition module, a background image management module, a background image compression module and a foreground video compression module; The monitoring decoding module receives the original video compression data and the corresponding national standard encoding collected by the enterprise end, decodes the original video compression data, and constructs the monitoring video; The surveillance video foreground acquisition module extracts video streams from the surveillance video in sequence according to the preset time window, takes the first video frame in the video stream as the background image or obtains the most recent background image from the background image management module; uses the image segmentation algorithm to perform background segmentation on each video frame in each video stream, and extracts several foreground YUV video streams; The foreground video compression module compresses each foreground YUV video stream, determines whether adjacent foreground YUV video streams are continuous based on the start recording time, and packages multiple continuous compressed foreground YUV video streams into a complete foreground video stream; The monitoring video background acquisition module compares each video frame in the monitoring video according to the foreground YUV video stream, selects the video frame with the smallest comparison difference value as the background image and transmits it to the background image management module; the background image includes the Class A background image configured by the enterprise end and transmitted according to the network idle sending rule, and the Class B background image transmitted in real time; A background image management module receives the background image, saves it, and sends it to the background compression module; A background image compression module compresses the background image, generates a compressed background image and sends it to the monitoring data encryption module; The monitoring data encryption module encrypts the compressed background image using a preset encryption algorithm, obtains the compressed ciphertext of the background image and sends it to the background image sending module; The complete foreground video stream is encrypted using a preset encryption algorithm, and the compressed ciphertext of the foreground video stream is obtained and sent to the enterprise end.

4. The video compression transmission system based on multi-level video cascade monitoring according to claim 1 is characterized in that: The background image service module includes a background image receiving module and a background image storage module; The background image receiving module receives the background image compressed ciphertext transmitted by the background image sending module, and transmits it to the background image storage module for storage, or transmits it to the group headquarters according to the video playback request.

5. The video compression transmission system based on multi-level video cascade monitoring according to claim 3 is characterized in that: The video decoding module includes a monitoring data decryption module and a video decoding synthesis module; The monitoring data decryption module uses a preset decryption algorithm to decrypt the compressed ciphertext of the foreground video stream to obtain the complete foreground video stream, and decrypt the compressed ciphertext of the background image to obtain the compressed background image; The video decoding and synthesis module decodes the complete foreground video stream to obtain the foreground YUV video stream; Decoding the compressed background image to obtain a background image; Each foreground video frame in the foreground YUV video stream is merged with the background image, the content in the foreground video frame covers the pixels at the corresponding position in the background image, and the background image with the closest time is added between the two foreground YUV video streams to obtain a complete video stream and send it to the group headquarters.

6. The video compression transmission system based on multi-level video cascade monitoring according to claim 3 is characterized in that: The process of extracting the foreground YUV video stream by the surveillance video foreground acquisition module is as follows: Step 11: extracting video streams from the surveillance video in sequence according to the preset time window, taking the first video frame in the video stream as a reference background image, or retrieving the most recent background image from the background image management module as a reference background image; Step 12: Use an image segmentation algorithm to segment each video frame in the video stream, extract the foreground image and construct a foreground YUV video stream; Step 121: Segment one or more initial foreground images from each video frame, mark the maximum range of all initial foreground images in the four directions of up, down, left, and right in the reference background image, and obtain four sets of coordinate positions (a1, a2, b1, b2); Step 122: construct a rectangle that completely covers the four sets of coordinate positions, and obtain the coordinates of the four corner points of the rectangle (c1, c2, d1, d2); Step 123: extracting the image within the rectangular range as the foreground image, and the foreground images of all video frames form a foreground YUV video stream; Step 13: extracting an image within a rectangular range from the reference background image as a motion judgment background; Step 14: Calculate the difference between each foreground image and the motion judgment background in the foreground YUV video stream as the foreground change value. When there is at least one set of foreground change values ​​greater than or equal to the set motion threshold Q, it is determined that there is motion change in the foreground YUV video stream, and enter step 15; Otherwise, there is no motion change in the foreground YUV video stream, and the process goes to step 16; Step 15: Send the foreground YUV video stream and the corresponding start recording time, the coordinate position relative to the reference background image, and the national standard code to the foreground video compression module, extract the video stream of the next continuous time period for processing, and return to step 12; Step 16: Abandon the current foreground YUV video stream, extract the video stream of the next continuous time period for processing, and return to step 12.

7. The video compression transmission system based on multi-level video cascade monitoring according to claim 6 is characterized in that: The specific process of extracting background images by the surveillance video background acquisition module is as follows: Step 21: retrieve the most recent background image from the background image management module as a reference background image, and remove pixels within a rectangular range in the reference background image to obtain a removed reference background image; Step 22: Eliminate pixels within a rectangular range in each video frame in the monitoring video to obtain a eliminated video frame; Step 23: Calculate the difference between the removed video frame and the removed reference background image corresponding to each video frame as the background change value; Step 24: when the background change value is less than the preset background small change threshold, it is determined that the background change of the current video frame relative to the reference background image is small, and the current video frame is discarded; When the background change value is greater than or equal to the preset background small change threshold, and less than or equal to the preset background large change threshold, the current video frame is marked as a Class A background image, and the Class A background image, recording time and national standard code are sent to the background image management module; When the background change value is greater than the background large change threshold, the current video frame is marked as a Class B background image, and the video frame, recording time and national standard code corresponding to the Class B background image are sent to the background image management module.

8. The video compression transmission system based on multi-level video cascade monitoring according to claim 6 is characterized in that: When the background image management module sends the background image, the first Class A background image and the corresponding recording time and national standard code are sent to the background image compression module in real time, and the remaining Class A background images and the corresponding recording time and national standard code are sent to the background image compression module according to the network idle sending rules configured on the enterprise side, and only the most recent Class A background image is saved; the Class B background image and the corresponding recording time and national standard code are sent to the background image compression module in real time, and only the most recent Class B background image is saved.

9. The video compression transmission system based on multi-level video cascade monitoring according to claim 7 is characterized in that: When the video intelligent compression module is run for the first time, the background image management module does not store the background image. Before the video intelligent compression module is run for the first time, a background image is collected during non-working hours and saved in the background image management module. The specific process is as follows: The monitoring decoding module sends a set number of video frames to the monitoring video background acquisition module. The monitoring video background acquisition module calculates the difference values ​​between all video frames. When the difference value is less than the difference threshold P configured on the enterprise side, the last video frame is used as a Class A background image, and the Class A background image and recording time are sent to the background image management module; when all difference values ​​are greater than or equal to the difference threshold P, the monitoring video background acquisition module continues to receive new video frames and repeatedly calculates the difference values ​​until the difference value is less than the difference threshold P.

10. A video compression transmission method based on multi-level video cascade monitoring, characterized in that: A video compression transmission system based on multi-level video cascade monitoring as described in any one of claims 1 to 9 comprises the following steps: Step 1: Perform background segmentation, compression and encryption on the surveillance videos collected by the enterprise terminals at all levels to generate background image compression ciphertext and foreground video stream compression ciphertext; the enterprise terminal saves the foreground video stream compression ciphertext and transmits the background image compression ciphertext to the group headquarters; Step 2: The group headquarters receives the compressed ciphertext of the background image and saves it; Step 3: The group headquarters generates a video playback request and transmits it to the corresponding enterprise end, and receives the compressed ciphertext playback address of the foreground video stream returned by the enterprise end; Step 4: The group headquarters pulls the compressed ciphertext of the foreground video stream from the corresponding enterprise end according to the compressed ciphertext playback address of the foreground video stream, and at the same time retrieves the compressed ciphertext of the background image according to the video playback request; Step 5: Decrypt, decode and merge the retrieved background image compressed ciphertext and foreground video stream compressed ciphertext to obtain a complete video stream for playback.

Images