Region-aware video transcoding system and method

Abstract

The application discloses a video hierarchical coding system and method based on region perception, and the method comprises the following steps: a region perception stage: dividing the obtained monitoring video into video segments, processing the video frames based on the video segments, and identifying the region information of the subsequent continuous frames through the reference frame processing result fed back by the server; according to the application of the video, the frame set of each video segment is divided into multiple video subsegments of different levels in space; an encoding stage: using different configuration parameters to encode the video subsegments into video substreams of different qualities; a decoding stage: decoding the video substreams to restore the frames corresponding to the video subsegments, and merging the frames into video frames. The application fully considers the time redundancy and spatial redundancy of the video, greatly reduces the video bandwidth to be transmitted, and moreover, the realized region perception mechanism overcomes the limitation of the camera resources, not only meets the real-time processing demand of the video application, but also can be deployed on most cameras without GPU resources.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of video coding, and particularly relates to a video hierarchical coding system and method based on region perception. BACKGROUND

[0002] The video data generated by a large number of monitoring cameras deployed in cities and parks has great value and is widely used in public safety, traffic management, fire monitoring and other fields. Due to limited resources at the camera end, a typical video analysis pipeline is to unload data at the camera end to the cloud or edge server for processing by a high-power image processor (GPU). However, sending all data to the server will consume excessive network bandwidth, causing the communication infrastructure to be overburdened. In addition, the transmission of high data volume will also increase the network transmission delay and the inference delay of the server, making real-time processing more difficult. Therefore, it is necessary to provide a new video coding mechanism to greatly filter and compress video data to reduce transmission cost and processing delay on the premise of ensuring video analysis accuracy.

[0003] Currently, the work in the field of video analysis generally filters frames that do not change or are irrelevant to the application, and only encodes frames containing targets and uploads them to the server, which is called frame-level filtering method. However, the video application is interested in the region of interest (RoI) in the frame rather than the background region, and the frame-level filtering does not consider the distribution difference of the region and uniformly encodes using the same configuration parameter, resulting in additional transmission bandwidth and storage consumption. Although some researchers have proposed using a lightweight DNN (Deep Neural Network) model at the edge node to detect the RoI region and implement region-aware filtering (i.e. region-level filtering method) accordingly. However, running the DNN model requires a large amount of computing resources, and how to achieve real-time region perception on the camera node with limited resources is a major challenge.

[0004] Building upon the existing H.264 / 265 standards, some researchers have proposed Scalable Video Coding (SVC) and Region of Interest (RoI)-based Encoding. SVC can simultaneously encode multiple video streams of different qualities and transmit them over different links (such as WiFi and 5G). However, while SVC uses segmented coding for each video stream, it doesn't consider spatial redundancy caused by regional distribution within the frame. RoI-based encoding can apply different compression ratios to the Region of Interest and the background region. Existing surveillance cameras typically pre-specify RoI regions. However, in real-world scenarios, target movement causes RoI regions to change, making manual specification unsuitable for real-time video stream changes. Furthermore, RoI-based encoding only uses differentiated quantization parameters to control local coding quality within a frame region, without considering the impact of resolution on different regions, thus failing to fully utilize spatial redundancy.

[0005] In view of this, it is necessary to propose a region-aware video hierarchical encoding and decoding system and method, which combines a region-level filtering mechanism for video analysis. This system utilizes feedback information from a high-accuracy DNN model (such as Faster R-CNN or YOLO) on the server side to achieve real-time region awareness on resource-constrained edge nodes. Then, based on the specific video application, each video segment is spatially divided into multiple sub-segments of different levels, and these sub-segments are encoded into sub-streams of different qualities using different configuration parameters. Finally, the system is decoded on the server side to restore the original video frames. Summary of the Invention

[0006] To address the shortcomings of the existing technologies, the present invention aims to provide a region-aware video hierarchical encoding and decoding system and method, thereby solving the problem that real-time region awareness cannot be performed on resource-constrained camera nodes in the prior art. The present invention utilizes commonly used cameras without GPU resources, and can work in conjunction with a server (edge ​​or cloud server) to achieve real-time region awareness and video hierarchical encoding and decoding.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] The present invention provides a region-aware video hierarchical encoding and decoding system, comprising: edge nodes, a transmission network, and a server;

[0009] Edge nodes are used to acquire surveillance video data in real time. Based on area awareness, the video stream is hierarchically encoded into multiple video sub-streams and reference frames are set. The video sub-streams with specified reference frames are then sent to the server through the transmission network.

[0010] The server is configured to receive the hierarchical coded video sub-streams, complete decoding and synthesis of video frames, process reference frames by using a DNN model, and feed back the processing result to the edge node through a transmission network.

[0011] Further, the edge node comprises a camera, a first processor, an encoder, and a first communication interface.

[0012] The camera is configured to acquire monitoring video stream data.

[0013] The first processor is configured to perform region tracking and region identification on each frame of the monitoring video data by using the reference frame processing result sent by the server, divide the video segment into a plurality of video sub-segments, and specify reference frames.

[0014] The encoder is configured to encode the plurality of video sub-segments into video sub-streams of different qualities by using different configuration parameters.

[0015] The first communication interface is configured to send the video sub-stream of the specified reference frame to the server, and receive the reference frame processing result sent by the server.

[0016] Further, the server comprises a decoder, a second processor, a graphics processor, and a second communication interface.

[0017] The graphics processor is configured to process the reference frames by using a DNN model to obtain a reference frame processing result.

[0018] The decoder is configured to decode the video sub-streams into video sub-segments.

[0019] The second processor is configured to merge the video sub-segments into video frames according to frame sequences, receive the reference frame processing result, and package the reference frame processing result.

[0020] The second communication interface is configured to receive the video sub-streams sent by the edge node, and send the packaged reference frame processing result to the edge node.

[0021] The video hierarchical coding method based on region perception comprises a region perception stage, an encoding stage, and a decoding stage.

[0022] In the region perception stage, the acquired monitoring video is divided into video segments, video frames are processed based on the video segments, region information of subsequent continuous frames is identified by using the reference frame processing result fed back by the server, and frame sets of each video segment are divided into a plurality of video sub-segments of different levels in space according to the application of the video.

[0023] In the encoding stage, the video sub-segments are encoded into video sub-streams of different qualities by using different configuration parameters.

[0024] Decoding stage: decode the video substream to restore the frames corresponding to the video subsegment, and merge them into video frames.

[0025] Further, the specific steps of the region awareness stage are:

[0026] (11) Initially, the edge node sends a certain time of monitoring video data to the server, the server establishes a background image, and sends the background image to the edge node;

[0027] (12) The edge node divides the monitoring video into video segments, and performs difference operation on each frame in the video segment and the background image to generate a difference image; then extracts the outer contour of the difference image to generate a potential target region;

[0028] (13) The edge node marks the RoI through region tracking according to the reference frame processing result sent by the server; merges the adjacent regions of the frame based on the set relative distance threshold, and generates multiple video subsegments.

[0029] Further, the step (11) specifically comprises:

[0030] The server receives the monitoring video data sent by the edge node, establishes a background image based on a background modeling algorithm; uses a DNN model to perform target detection on the background image, and replaces the background region at the corresponding position in the sequence frame for the region with a target; sends the background image to the edge node;

[0031] Wherein, the background image is periodically updated or trigger updated; for trigger update, the server checks the region with a target in the background image based on the reference frame processing result in each video segment processing period, and if the target moves, the corresponding region of the reference frame is used to update the background image.

[0032] Further, the step (12) specifically comprises:

[0033] (121) The first processor groups the buffered frames into a video segment based on a set processing period, and specifies a reference frame; the set of all frames in the video segment is denoted as F seg ;

[0034] (122) Each frame in the video segment is reduced in resolution by downsampling, and the background image is subjected to the same downsampling operation; each frame is subjected to difference operation with the background image to generate a difference image;

[0035] (123) For all difference images, generate an outer contour, and delete the outer contour smaller than an area threshold; for the stationary target in the video frame, use the target detection result of the background image to generate a corresponding region on the current frame to realize the generation of all potential target regions in the video segment.

[0036] Further, the step (13) specifically comprises:

[0037] (131) Target detection on the reference frame by using the DNN model to identify the region information where the target is located, denoted as the processing result R dnn ; and sending the processing result R dnn to the corresponding edge node;

[0038] (132) If the set of potential target regions generated based on the differential operation of the reference frame is R ref , then perform RoI marking based on the region overlap degree. Let any two regions be r1 and r2, and define the region overlap degree O of r1 and r2 as follows:

[0039]

[0040] In the formula, area(r1∩r2) represents the area of the intersection region, and min(area(r1), area(r2)) represents the minimum value of the areas of the two regions.

[0041] The i-th region of R ref is The j-th region of R dnn is The region overlap degree threshold is T, and r i ref is set to be Then is defined as follows:

[0042]

[0043] Any region of R ref is marked as RoI according to , and R ref is divided into two types of sub-regions: RoI region and NRoI region. After RoI marking, R ref is denoted as R ref* .

[0044] (133) Based on the marking result of the reference frame, perform RoI marking on the regions of all frames of the current video segment by using region tracking. Merge the adjacent regions of each video frame based on the set relative distance threshold, and generate multiple video sub-segments.

[0045] Further, the step (133) specifically comprises:

[0046] (1331) Region tracking on the sequence frames in the video segment based on the region overlap degree. Take the reference frame with R ref* as the current frame, and continuously perform region tracking on the subsequent frames through the RoI marking of the previous frame to perform RoI marking on the regions of all frames of the current video segment.

[0047] (1332)merge the adjacent regions with RoI label in the video frame based on the set relative distance threshold, and then divide all the video frames in the current video segment into different video sub-segments based on the RoI label, at least containing RoI sub-segment and NRoI sub-segment and NRoI sub-segment

[0048] Further, the specific steps of the encoding stage are:

[0049] (21) The server configures the monitoring video data for a certain time to generate a configuration table about resolution, quantization parameter and accuracy, and sends the configuration table to the edge node.

[0050] (22) The edge node selects configuration parameters for hierarchical encoding, encodes the divided video sub-segments into video sub-streams of different qualities, and sends the encoded video sub-streams to the server.

[0051] Further, the specific steps of the step (21) are:

[0052] The server configures the monitoring video data for a certain time to generate a configuration table about resolution, quantization parameter and accuracy, and sends the configuration table to the edge node.

[0053] Wherein, the configuration profile is updated periodically or triggered; for triggered update, the server detects the brightness and data distribution of each video segment, and if the detection result exceeds a certain threshold, the edge node is notified to send the video data for a certain time again, and the configuration profile is configured again.

[0054] Further, the specific steps of the step (22) are:

[0055] (221) The first processor selects table entries from the configuration table based on the accuracy of the target to encode the video sub-segment: RoI sub-segment For video analysis, high-quality configuration parameters are used to encode RoI sub-stream; NRoI sub-segment For video analysis, low-quality configuration parameters are used to encode NRoI sub-stream;

[0056] (222)Based on the reference frame processing result, it is evaluated whether the configuration parameters required for evaluating the current video segment meet the accuracy requirement, and under a certain target detection confidence threshold, if the number of targets below the confidence threshold in the reference frame is more, high configuration parameters are set to improve the coding quality, otherwise, if the confidence of the targets in the reference frame is higher than the confidence threshold, low configuration parameters are set to reduce the coding quality.

[0057] Further, the specific steps of the decoding stage are:

[0058] (31) decoding the video substream into the corresponding video subsegment, and then up-sampling all the low-quality video subsegments to the resolution of the highest-quality video subsegment;

[0059] (32) merging the up-sampled video subsegments according to the frame sequence to generate a video frame set Γ seg , the merging operation is performed by pixel addition or maximum pixel method, and the specific steps are as follows:

[0060]

[0061] In the formula, the operator represents the pixel addition or maximum pixel operation, and the ellipsis represents multiple levels of subsegments, represents the decoded RoI subsegment, represents the decoded NRoI subsegment.

[0062] Advantages of the present application:

[0063] The present application fully considers the temporal redundancy and spatial redundancy of the video, greatly reduces the transmitted video bandwidth; in addition, the realized region perception mechanism overcomes the limitation of camera resources, not only meets the real-time processing demand of video application, but also can be deployed on most cameras without GPU resources.

[0064] 1、The present application uses different configuration parameters in the edge node to encode multiple video subsegments into substreams of different qualities, and then decodes and restores them into original video frames in the server end. The encoding mechanism fully utilizes the spatial redundancy and temporal redundancy of the video, and can greatly reduce the transmitted video bandwidth. Compared with the standard segmented encoding mechanism, when transmitting high-definition traffic monitoring video, setting the segment length to 10 frames, the video hierarchical coding and decoding mechanism can save up to 40% of the bandwidth (the accuracy of video analysis decreases by no more than 3%).

[0065] 2、The present application uses only low-level video features in the edge node to divide each video segment into multiple subsegments of different levels in space through the processing result of the DNN model, not only meets the real-time processing demand of video application, but also overcomes the limitation of camera resources, and can be deployed on most cameras without GPU.

[0066] 3、The application can continuously perceive changes in video content and link state through configuration images and area evaluation mechanisms when selecting encoding parameters of sub-sections, and can adaptively adjust the encoding parameters (resolution, quantization parameter, etc.) of the sub-sections, which is particularly suitable for variable edge heterogeneous environments. BRIEF DESCRIPTION OF DRAWINGS

[0067] Figure 1 The figure is a system architecture diagram of the application;

[0068] Figure 2 The figure is a flowchart of the method of the application. DETAILED DESCRIPTION

[0069] For the convenience of those skilled in the art, the application will be further described below in conjunction with the embodiments and the drawings, and the content mentioned in the embodiments is not a limitation of the application.

[0070] Referring to Figure 1 The video transcoding system based on area perception of the application includes an edge node, a transmission network, and a server.

[0071] The edge node is used to acquire monitoring video data in real time, encode the video stream into multiple video sub-streams based on area perception and set reference frames, and then send the video sub-streams with specified reference frames to the server through the transmission network.

[0072] The edge node includes a camera, a first processor, an encoder, and a first communication interface.

[0073] The camera is used to collect monitoring video stream data.

[0074] The first processor uses the reference frame processing result sent by the server to perform area tracking and area identification on each frame of the monitoring video data, divides the video section into multiple video sub-sections, and specifies the reference frames.

[0075] The encoder is used to encode the multiple video sub-sections into video sub-streams of different qualities using different configuration parameters.

[0076] The first communication interface is used to send the video sub-streams with specified reference frames to the server and receive the reference frame processing result sent by the server.

[0077] The server is used to receive the video sub-streams after transcoding, complete decoding and synthesis of video frames, process the reference frames through a DNN model, and feed back the processing result to the edge node through the transmission network.

[0078] The server includes a decoder, a second processor, a graphics processing unit (GPU), and a second communication interface.

[0079] a graphics processor configured to process the reference frame by using the DNN model to obtain a reference frame processing result;

[0080] a decoder configured to decode the video substream and decode the video substream into a video subsegment;

[0081] a second processor configured to merge the video subsegment, merge the plurality of video subsegments into a video frame according to a frame sequence, receive the reference frame processing result, and package the reference frame processing result;

[0082] a second communication interface configured to receive the video substream sent by the edge node and send the packaged reference frame processing result to the edge node.

[0083] Referring to Figure 2 The video hierarchical coding method based on region perception provided by the application comprises a region perception stage, an encoding stage and a decoding stage, wherein,

[0084] The region perception stage comprises the following steps:

[0085] (11) Initially, the edge node sends monitoring video data of a certain time (e.g., 5 minutes) to the server, the server establishes a background image, and sends the background image to the edge node;

[0086] (12) The edge node divides the monitoring video into video segments, performs difference operation on each frame in the video segment and the background image to generate a difference image, and extracts the outer contour of the difference image to generate a potential target region;

[0087] (13) The edge node marks the RoI by region tracking according to the reference frame processing result sent by the server, merges the adjacent regions of the frames based on a set relative distance threshold, and generates a plurality of video subsegments.

[0088] The step (11) specifically comprises:

[0089] The server receives the monitoring video data sent by the edge node, establishes a background image based on a background modeling algorithm (e.g., median method), performs target detection on the background image by using a DNN model (e.g., YOLOv5), replaces the background region at the corresponding position in the sequence frame for the region with a target, and sends the background image to the edge node;

[0090] Wherein, the background image is periodically updated (such as every 1 hour) or triggered to update; for triggered update, the server checks the area where the target exists in the background image based on the reference frame processing result in each video segment processing period, and if the target moves, the corresponding area of the reference frame is used to update the background image.

[0091] Wherein, the step (12) specifically comprises:

[0092] (121) The first processor groups the cache frames into a video segment based on a set processing period (such as 3s), and specifies a reference frame (generally the last frame of the current segment); the set of all frames in the video segment is denoted as F seg ;

[0093] (122) Each frame in the video segment is reduced in resolution by downsampling (for example, the resolution is reduced from 1920x1080 to 640x360), and the background image is subjected to the same downsampling operation; each frame is subjected to difference operation with the background image to generate a difference image;

[0094] (123) For all difference images, an outer contour is generated, and the outer contour smaller than an area threshold is deleted; for a stationary target in the video frame, the target detection result of the background image is used to generate a corresponding area on the current frame to realize the generation of all potential target areas in the video segment.

[0095] Wherein, the step (13) specifically comprises:

[0096] (131) The reference frame is subjected to target detection using a DNN model to identify the area information (bounding box, classification, confidence, etc.) where the target is located, denoted as processing result R dnn ; the processing result R dnn is sent to the corresponding edge node;

[0097] (132) The set of potential target areas generated based on difference operation of the reference frame is R ref , then the RoI is labeled based on the area overlap degree; set any two areas as r1 and r2, and define the area overlap degree O of r1 and r2 as:

[0098]

[0099] In the formula, area(r1∩r2) represents the area of the intersection, and min(area(r1), area(r2)) represents the minimum value of the areas of the two areas;

[0100] The i-th area of R ref is The j-th area of R dnn is The region overlap threshold is T (typically set to 0.5). RoI is marked as but The definition is as follows:

[0101]

[0102] R ref any region according to Perform RoI tagging, R ref The regions are divided into two sub-regions: RoI regions and NRoI regions; after RoI labeling, the R... ref Noted as R ref* ;

[0103] (133) Based on the labeling results of the reference frame, the region of all frames in the current video segment is labeled with RoI using the region tracking method; the neighboring regions of each video frame are merged based on the set relative distance threshold, and multiple video segments are generated.

[0104] Specifically, step (133) includes:

[0105] (1331) Based on the region overlap, perform region tracking on the sequence frames within the video segment, and select the region with R ref* The reference frame is used as the current frame, and subsequent frames continuously perform region tracking by using the RoI mark of the previous frame, marking the regions of all frames in the current video segment with RoI;

[0106] (1332) Merge neighboring regions with RoI tags in video frames based on a set relative distance threshold, and then divide all video frames in the current video segment into different video segments based on RoI tags, including at least RoI segments. and NRoI subfield

[0107] For example, for general video applications such as traffic monitoring, F seg Divided into two video segments: RoI segment and NRoI subfield

[0108] For specific video applications such as pedestrian recognition, F seg Divided into three video segments: pedestrian segment Other target segments and NRoI subfield

[0109] Encoding stage: Video segments are encoded into video substreams of different qualities using different configuration parameters; the specific steps are as follows:

[0110] (21) The server configures the video data of a certain time to generate a configuration table about resolution, quantization parameter (QP), and accuracy, and sends the configuration table to the edge node;

[0111] (22) The edge node selects configuration parameters (such as resolution and quantization parameter) for hierarchical coding, encodes the divided video sub-segments into video sub-streams of different qualities, and sends the encoded video sub-streams to the server.

[0112] The specific steps of step (21) are:

[0113] The server generates label data (ground truth) by configuring the video data of a certain time through a DNN model, obtains video data of different qualities by combining resolution and quantization parameter, and detects the accuracy of the video data by using a DNN model to obtain a configuration table.

[0114] The configuration profile is periodically updated (e.g., every 4 hours) or triggered. For triggered updates, the server detects the brightness (e.g., changes in sunny and cloudy weather) and data distribution (e.g., too many small targets) of each video segment. If the detection result exceeds a certain threshold, the edge node is notified to send video data of a certain time again, and the configuration profile is configured again.

[0115] The specific steps of step (22) are:

[0116] (221) The first processor selects a table entry from the configuration table based on the accuracy of the video sub-segment: RoI sub-segment For video analysis, high-quality configuration parameters are used to encode RoI sub-streams; NRoI sub-segment For video analysis, low-quality configuration parameters are used to encode NRoI sub-streams;

[0117] For example, in the case of transmitting high-definition video, RoI sub-segments are encoded into RoI sub-streams using high configuration s1920x1080qp26 (resolution 1920x1080, quantization parameter 26); for NRoI sub-segments are encoded into NRoI sub-streams using low configuration s640x360qp30 (resolution 640x360, quantization parameter 30).

[0118] (222)Based on the reference frame processing result, whether the configuration parameters required for evaluating the current video segment meet the accuracy requirement, under a certain target detection confidence threshold (the default value is 0.3), if the number of targets below the confidence threshold in the reference frame is more, set a high configuration parameter to improve the encoding quality; on the contrary, if the confidence of the target in the reference frame is higher than the confidence threshold, set a low configuration parameter to reduce the encoding quality.

[0119] Decoding stage: decode the video substream to restore the frame corresponding to the video subsegment, and merge into a video frame; the specific steps are as follows:

[0120] (31) Decode the video substream into the corresponding video subsegment, and then upsample all the low-quality video subsegments to the resolution of the highest-quality video subsegment;

[0121] For example, taking receiving a high-definition video as an example, after the server receives the RoI substream and the NRoI substream in step (221), first decode the RoI substream and the NRoI substream into RoI subsegment and NRoI subsegment Then upsample the resolution of each frame of from 640x360 to 1920x1080, denoted as

[0122] (32) Merge the upsampled video subsegment according to the frame sequence to generate a video frame set seg The merging operation is performed by pixel addition or maximum pixel method, and the specific steps are as follows:

[0123]

[0124] In the formula, the operator represents the pixel addition or maximum pixel operation, the ellipsis represents multiple levels of subsegments, represents the decoded RoI subsegment, represents the decoded NRoI subsegment.

[0125] The present application has many specific application approaches, and the above description is only the preferred embodiment of the present application. It should be pointed out that for ordinary skilled persons in the technical field, without departing from the principle of the present application, a number of improvements can be made, and these improvements should also be considered as the protection scope of the present application.

Claims

1. A region-aware video hierarchical encoding and decoding system, characterized in that, include: Edge nodes, transmission networks, and servers; Edge nodes are used to acquire surveillance video data in real time. Based on area awareness, the video stream is hierarchically encoded into multiple video sub-streams and reference frames are set. The video sub-streams with specified reference frames are then sent to the server through the transmission network. The server receives the hierarchically encoded video sub-streams, decodes and synthesizes video frames; it processes reference frames using a DNN model and feeds the processing results back to the edge nodes via the transmission network.

2. The region-aware video hierarchical coding and decoding system according to claim 1, characterized in that, The edge node includes: a camera, a first processor, an encoder, and a first communication interface; Cameras are used to collect surveillance video stream data; The first processor uses the processing results of the reference frames sent by the server to perform region tracking and region identification on each frame in the surveillance video data, and divides the video segment into multiple video sub-segments and specifies the reference frames. An encoder is used to encode multiple video segments into video substreams of different qualities using different configuration parameters; The first communication interface is used to send the video sub-stream of the specified reference frame to the server and receive the reference frame processing result sent by the server. The server includes: a decoder, a second processor, a graphics processor, and a second communication interface; A graphics processor is used to process the reference frame using a DNN model to obtain the processing result of the reference frame. A decoder is used to decode a video substream into video segments. The second processor is used to merge video segments, combining multiple video segments into a video frame according to the frame sequence; it also receives the processing results of the reference frame and encapsulates them. The second communication interface is used to receive video sub-streams sent by edge nodes and send the encapsulated reference frame processing results to the edge nodes.

3. A region-aware video hierarchical encoding and decoding method, based on the system described in any one of claims 1-2, characterized in that, The methods include: The region sensing stage, the encoding stage, and the decoding stage; Area perception stage: The acquired surveillance video is divided into video segments, video frames are processed based on the video segments, and the area information of subsequent continuous frames is identified by the processing results of reference frames fed back by the server. According to the application of the video, the frame set of each video segment is spatially divided into multiple video sub-segments of different levels. Encoding phase: Video segments are encoded into video substreams of different qualities using different configuration parameters; Decoding stage: Decode the video substreams to restore them to the frames corresponding to the video segments, and merge them into video frames.

4. The method according to claim 3, characterized in that, The specific steps of the area perception phase are as follows: (11) Initially, the edge nodes send monitoring video data to the server for a certain period of time. The server creates a background image and sends the background image to the edge nodes. (12) The edge node divides the monitoring video into video segments, performs a difference operation between each frame in the video segment and the background image, and generates a difference image; Then, the outer contour is extracted from the difference image to generate potential target regions; (13) Edge nodes perform RoI marking through region tracking based on the processing results of the reference frames sent by the server; merge the neighboring regions of the frames based on the set relative distance threshold, and generate multiple video segments.

5. The method according to claim 4, characterized in that, Step (11) specifically includes: The server receives monitoring video data sent by the edge nodes, builds a background image based on a background modeling algorithm, performs target detection on the background image using a DNN model, and replaces the background region at the corresponding position in the sequence frame with the region containing the target; then sends the background image to the edge nodes. The background image is updated periodically or triggered. For triggered updates, the server checks the area in the background image where the target exists based on the processing result of the reference frame during the processing cycle of each video segment. If the target moves, the background image is updated using the corresponding area of ​​the reference frame. Step (12) specifically includes: (121) Edge nodes assemble cached frames into video segments based on a set processing cycle and specify a reference frame; the set of all frames within a video segment is denoted as F. seg ; (122) Reduce the resolution of each frame in the video segment by downsampling, and perform the same downsampling operation on the background image; perform a difference operation between each frame and the background image to generate a difference image; (123) Generate outer contours for all difference images and delete outer contours smaller than the area threshold; for stationary targets in video frames, use the target detection results of the background image to generate the corresponding region on the current frame, so as to generate all potential target regions in the video segment. Step (13) specifically includes: (131) Use a DNN model to perform target detection on the reference frame, identify the region information where the target is located, and denot the processing result R. dnn The processing result R dnn Send it to the corresponding edge node; (132) The set of potential target regions generated by the reference frame based on the difference operation is R. ref Then, RoI labeling is performed based on the region overlap; let any two regions be r1 and r2, and define the region overlap O of r1 and r2 as: In the formula, area(r1∩r2) represents the area of ​​the intersecting region, and min(area(r1),area(r2)) represents the minimum area of ​​the two regions; R ref The i-th region is r i ref R dnn The j-th region is The region overlap threshold is T, and r is set. i ref RoI is marked as but The definition is as follows: R ref any region according to Perform RoI tagging, R ref The regions are divided into two sub-regions: RoI regions and NRoI regions; after RoI labeling, the R... ref Noted as R ref* ; (133) Based on the labeling results of the reference frame, the region of all frames in the current video segment is labeled with RoI using the region tracking method; the neighboring regions of each video frame are merged based on the set relative distance threshold, and multiple video segments are generated.

6. The method according to claim 5, characterized in that, The specific steps (133) include: (1331) Based on the region overlap, perform region tracking on the sequence frames within the video segment, and select the region with R ref* The reference frame is used as the current frame, and subsequent frames continuously perform region tracking by using the RoI mark of the previous frame, marking the regions of all frames in the current video segment with RoI; (1332) Merge neighboring regions with RoI tags in video frames based on a set relative distance threshold, and then divide all video frames in the current video segment into different video segments based on RoI tags, including at least RoI segments. and NRoI subfield 7. The method according to claim 4, characterized in that, The specific steps of the encoding stage are as follows: (21) The server configures the profile of the surveillance video data for a certain period of time, generates a configuration table for resolution, quantization parameters and accuracy, and sends the configuration table to the edge node. (22) Edge nodes select configuration parameters for hierarchical encoding, encode multiple video segments into video sub-streams of different quality, and send the encoded video sub-streams to the server.

8. The method according to claim 7, characterized in that, The specific steps of step (21) are as follows: The server uses a DNN model to create a configuration profile, generates tag data from the acquired surveillance video data over a certain period of time, and then obtains video data of different qualities by combining resolution and quantization parameters. The DNN model is then used to perform accuracy testing on the video data to obtain a configuration table. The configuration profile is updated periodically or triggered. For triggered updates, the server detects the brightness and data distribution of each video segment. If the detection result exceeds a certain threshold, the edge node is notified to send video data for a certain period of time again to reconfigure the profile.

9. The method according to claim 7, characterized in that, The specific steps of step (22) are as follows: (221) The first processor selects an entry from the configuration table based on the target accuracy to encode the video segment: RoI segment The accuracy of video analysis is greatly affected by the use of high-quality configuration parameters to encode it into RoI sub-streams and NRoI sub-segments. The impact on the accuracy of video analysis is minimal, so low-quality configuration parameters are used to encode it as an NRoI substream; (222) Based on the processing results of the reference frame, evaluate whether the configuration parameters required for the current video segment meet the accuracy requirements. Under a certain target detection confidence threshold, if there are more targets in the reference frame that are below the confidence threshold, set higher configuration parameters to improve the coding quality; conversely, if the confidence of the targets in the reference frame is higher than the confidence threshold, set lower configuration parameters to reduce the coding quality.

10. The method according to claim 7, characterized in that, The specific steps of the decoding stage are as follows: (31) Decode the video substream into corresponding video segments, and then upsample all low-quality video segments to increase their resolution to the highest quality video segment resolution. (32) Merge the upsampled video segments according to the frame sequence to generate a video frame set Γ. seg The merging operation is performed using either pixel addition or the maximum pixel method, as detailed below: In the formula, the operator ⊙ represents the operation of adding pixels or finding the maximum pixel value, and the ellipsis indicates that there are multiple levels of sub-segments. This represents the decoded RoI sub-segment. This represents the decoded NRoI sub-segment.

Images