Monitoring video data storage method with optimized data structure and video encoder

Abstract

The invention belongs to the technical field of data storage, and particularly relates to a monitoring video data storage method with an optimized data structure and a video encoder. The storage method comprises the following steps: S1, segmenting video stream data acquired by a front end into video clips with specified duration; s2, comparing and analyzing the start frame and the end frame of the video clip through a background difference algorithm, and generating a static data volume by using the video clip without the moving target; s3, any two adjacent frames in the video clip with the moving target are compared and analyzed through a background difference algorithm, and then a dynamic data volume is generated; s4, generating attribute segment contents of the dynamic data volume according to the data segment contents of the data elements in the dynamic data volume through a target recognition algorithm; and S5, temporarily storing the acquired static data body and the dynamic data body locally, and uploading the static data body and the dynamic data body to a data center at proper time. According to the method and the device, the problems that the existing monitoring type video data is high in storage space occupation and does not support content retrieval are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of data storage, and in particular relates to a monitoring video data storage method with optimized data structure, a video encoder for monitoring video data, and a video monitoring system. Background Art

[0002] A large number of security surveillance camera products have huge data storage requirements. For most surveillance cameras, the video content collected during most of the monitoring period is repeated. Storing this data will increase costs and waste storage space. To avoid this problem, technicians have proposed a variety of different technical routes to optimize storage space. One is to compress video data through post-processing. Another is to adjust parameters during the early video recording and use efficient encoding technology to reduce the amount of data.

[0003] Typical methods for optimizing storage space in the prior art include:

[0004] (1) Select an efficient video format: Common video formats such as MP4 have good compression performance and wide compatibility. (2) Adjust video parameters: Control the size and quality of video files by adjusting video parameters such as resolution, bit rate, frame rate, etc. Reducing the resolution, bit rate and frame rate can usually effectively reduce the size of video files. (3) Video compression algorithm: Traditional video compression algorithms, such as H.264 and H.265, follow a predictive coding structure and support manual design to achieve efficient compression performance. (4) Intra-frame compression (spatial compression): Similar to static image compression, it considers the data of the current frame without considering the redundant information between adjacent frames, and generally adopts a lossy compression algorithm. (5) Inter-frame compression (temporal compression): Utilize the redundant information of video frames in time and compress the data by comparing the data between different frames on the time axis, which is generally achieved through a frame difference algorithm. (5) Discrete cosine transform (DCT): Convert the image into a frequency domain signal, and then achieve compression by quantizing and encoding the frequency domain coefficients.

[0005] However, the data compression ratio of the existing optimized storage space is still insufficient, and most of the existing technologies need to compress the stored data within a certain period of time after the video stream data is stored locally. This will still greatly consume the service life of the memory. In addition, the existing video surveillance data does not support content retrieval, and can only be manually searched through video playback, so the use is relatively unchanged. Summary of the invention

[0006] In order to overcome the problems of high storage occupancy and difficulty in content search of existing surveillance video data, the present invention provides a surveillance video data storage method with optimized data structure, a video encoder for surveillance video data, and a video surveillance system.

[0007] The technical solution provided by the present invention is:

[0008] A monitoring video data storage method with optimized data structure comprises the following steps:

[0009] S1: Segment the video stream data collected by the front end into several video segments of specified length according to preset time intervals.

[0010] S2: A high-threshold background difference algorithm is used to compare and analyze the start and end frames of each video clip and make the following decisions:

[0011] (1) If there is a moving target in the video clip, a dynamic label is added to each video clip.

[0012] (2) If there is no moving target in the video clip, the start frame image is recorded as the image data of the data segment, and the time information from the start frame to the end frame is recorded as the static timestamp of the marker segment, thereby obtaining a static data body.

[0013] S3: Compare and analyze any two adjacent frames in the video clip with dynamic labels through a background difference algorithm with a low threshold, and make the following decisions:

[0014] (1) If there is a change between any two frames, the image data of the previous frame is retained as the image data of the data segment, and the time information of the previous frame is recorded as the dynamic timestamp of the marker segment, thereby obtaining a data element.

[0015] (2) If there is no change between any two frames, continue to compare the next frame until all frames in the video clip are identified.

[0016] All data elements obtained from each video segment with dynamic tags are arranged according to timestamps to form a dynamic data body of the current video segment.

[0017] S4: extracting image data from the data segment of each data element in the dynamic data body, inputting it into a target recognition algorithm, and adding an attribute segment for recording relevant information in each data element according to the recognition result.

[0018] S5: Temporarily store the acquired static data body and dynamic data body locally, and upload the temporarily stored data to the data center when the amount of stored data exceeds a preset capacity upper limit.

[0019] As a further improvement of the present invention, the data format of the static data body is as follows:

[0020] {data: {p0}, timestamp: {dy_x1, dy_x2}}

[0021] Among them, data represents the data segment, timestamp represents the marker segment; p0 represents the image data recorded in the data segment; dy_x1 and dy_x2 represent the timestamps corresponding to the start frame and end frame of the video segment corresponding to the static data body, respectively.

[0022] As a further improvement of the present invention, the data format of the dynamic data body is as follows:

[0023] {data: {p1}, timestamp: {stc_x1}, type: {c1},…, data: {pn}, timestamp: {stc_xn},

[0024] type: {cn}}

[0025] Among them, data represents the data segment, timestamp represents the marker segment; type represents the attribute segment; p1~pn represent the image data recorded in the data segments of the 1st to nth data elements respectively; stc_x 1~stc_x n represent the timestamps corresponding to the start frames of the 1st to nth data elements respectively; c1~cn represent the related information recorded in the attribute segments of the 1st to nth data elements respectively.

[0026] As a further improvement of the present invention, in step S1, the preset time interval does not exceed 5s.

[0027] As a further improvement of the present invention, in step S2 and step S3, the first frame image is used as the initial modeling background, and the image data recorded in the data segment of any previous static data body is used as the modeling background of the subsequent video segment with dynamic labels.

[0028] As a further improvement of the present invention, in step S4, the target recognition algorithm adopts a method based on R-CNN, Fast R-CNN, Faster R-CNN or YOLO.

[0029] As a further improvement of the present invention, in step S4, the relevant information recorded in the attribute segment of the dynamic data body includes the types of all moving objects contained in the identified image and their corresponding scene information.

[0030] The present invention also includes a video encoder for monitoring video data, which is used to optimize the data structure of the video stream data collected by the camera using the monitoring video data storage method with data structure optimization as described above. The video encoder includes: a data collection unit, a first encoding unit, a second encoding unit, and an attribute generation unit.

[0031] The data acquisition unit is used to obtain the video stream data collected by the front-end camera in real time, and generate video clips of specified length according to preset time intervals.

[0032] The first encoding unit is used to compare the start frame and the end frame of each video segment through a background difference algorithm with a high threshold value to see whether they have changed: if there is a change between the two, a dynamic tag is added to the video segment. If there is no change between the two, the start frame image is recorded as the image data of the data segment, and the time information from the start frame to the end frame is recorded as the static timestamp of the mark segment; thus, a static data body is obtained.

[0033] The second encoding unit is used to obtain the video segment with dynamic tags output by the first encoding unit. Then, a background difference algorithm with a low threshold is used to compare whether any two adjacent frames in the video segment with dynamic tags have changed: if there is a change between any two frames, the image data of the previous frame is retained as the image data of the data segment, and the time information of the previous frame is recorded as the dynamic timestamp of the marked segment, thereby obtaining a data element. If there is no change between any two frames, the next frame is compared until all frames in the video segment are identified. Finally, all data elements obtained from the video segment are arranged according to the timestamp to form the dynamic data body of the current video segment.

[0034] The attribute generation unit is used to extract the image data of the data segment in each data element of the dynamic data body, input it into a target recognition algorithm, and add an attribute segment for recording relevant information between the corresponding data element according to the recognition result.

[0035] As a further improvement of the present invention, the video encoder of monitoring video data further comprises a recompression unit, which is used to compress the image data recorded in the data segments of the static data body and the dynamic data body by using a spatial compression or DCT method.

[0036] The present invention also includes a video monitoring system, which includes a local device and a remote data center; the data center is used to permanently store the video data uploaded by the local device and respond to query requests for video content from authorized users.

[0037] The local devices in the video monitoring system include: a camera, a data processing module and a storage module. The camera is used to collect the original video stream data in the target area.

[0038] The data processing module runs a video encoder for monitoring video data as described above. The video encoder is used to obtain the original video stream data collected by the camera and optimize its data structure to obtain new video stream data composed of static data body and dynamic data body.

[0039] The storage module includes a memory and a storage management unit. The storage management unit is used to store original video stream data and / or new video stream data in the memory, and when the remaining capacity of the memory is lower than a preset capacity lower limit, upload the data stored in the memory to a background data center.

[0040] The technical solution provided by the present invention has the following beneficial effects:

[0041] The present invention provides a new data storage format that can optimize the storage space of surveillance video data, and designs a transcoding method between the original video stream data and the new video stream data format. In the new scheme of the invention, static and dynamic storage resources are greatly utilized before data storage, and a new storage structure composed of dynamic data bodies and static data bodies is obtained through edge computing. This scheme can greatly optimize the size of the video stream data finally stored in the data center while maintaining the picture quality, thereby achieving the purpose of optimizing the storage space.

[0042] In the new surveillance video stream data storage format provided by the invention, extended attributes containing more descriptive information are also provided in the dynamic data body. The scene description, target type and other information in the extended attribute content can lay the foundation for subsequent video stream analysis, video stream interception, control, etc., and provide convenience for users to retrieve video content. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A flowchart of the steps of a monitoring video data storage method with optimized data structure provided in Example 1 of the present invention.

[0044] Figure 2 This is the data format of the static data body and the dynamic data body designed in Embodiment 1 of the present invention for encoding video stream data.

[0045] Figure 3 These are the types of video clips corresponding to different dynamic data bodies in Example 1 of the present invention.

[0046] Figure 4 This is a flow chart of a method for generating a dynamic structure provided in Example 1 of the present invention.

[0047] Figure 5 A module diagram of a video encoder for monitoring video data provided in Example 2 of the present invention.

[0048] Figure 6 A module diagram of a video encoder with a recompression unit provided in Embodiment 2 of the present invention.

[0049] Figure 7 This is a system topology diagram of the video surveillance system provided in Example 3 of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0051] Example 1

[0052] In order to solve the problems that the existing surveillance video data storage capacity is large and does not support content query, this embodiment designs a new video data format, which stores a video clip of a specified length through a special static data body and a dynamic data body. This embodiment also provides a transcoding method for transcoding the video stream data collected by the existing surveillance camera into the new video data format, thereby obtaining a surveillance video data storage method with optimized data structure.

[0053] Compared with the existing solutions, the new surveillance video data storage method provided by this embodiment has three major advantages: First, the storage space occupied by video data can be compressed to the maximum extent without affecting the image quality. Second, real-time transcoding can be performed while the video is being recorded or during the video stream push process, without the need to store unprocessed data on the local storage device, thereby reducing the life loss of the local storage device. Third, attribute information related to the video content is added to the storage format of the new video data, thereby supporting users to achieve content retrieval without playing the relevant pictures.

[0054] Specifically, Figure 1 As shown, the monitoring video data storage method with optimized data structure provided in this embodiment includes the following steps:

[0055] S1: Segment the video stream data collected by the front end into several video segments of specified length according to preset time intervals.

[0056] In the scheme of the present embodiment, the basic unit of video data storage is a data body, which can be divided into a dynamic data body and a static data body. The content of each dynamic data body or static data body in the present embodiment corresponds to a video segment of a specified duration in the original video stream data. Therefore, in order to realize transcoding between data, the collected original video stream data needs to be divided into corresponding video segments according to the specified time interval. The segmentation operation of the original video stream data can be implemented while the video is being recorded or during the video stream push process. For example, assuming that the preset time interval is 5s, a real-time video segment can be generated every 5s during video recording, or the recorded video can be segmented in units of 5s in duration.

[0057] In this embodiment, the length of the preset time interval will affect the compression rate of the new video data after transcoding. Generally speaking, the longer the preset time interval is, the higher the compression rate of the new video data composed of the dynamic data body or the static data body is, but a higher compression rate may also cause the loss of some information (the compression of the file by the scheme of this embodiment will not affect the image quality). Therefore, in practical applications, users can flexibly select the size of the time interval according to actual needs. For example, in scenes such as daytime when there are many targets in the monitoring area or the target movement is more intense, the time interval can be set to be relatively short, while in scenes such as nighttime when there are fewer targets in the monitoring area or the target movement is slower, the time interval can be increased to appropriately extend the duration of each segmented video segment. Generally speaking, in most application scenarios, the preset time interval does not exceed 5s.

[0058] S2: Use a background difference algorithm with a high threshold to compare and analyze the start and end frames of each video clip, and then make the following decisions based on the results:

[0059] (1) If there is a moving target in the video clip, a dynamic label is added to each video clip.

[0060] (2) If there is no moving target in the video clip, the start frame image is recorded as the image data of the data segment, and the time information from the start frame to the end frame is recorded as the static timestamp of the marker segment, thereby obtaining a static data body.

[0061] This embodiment uses a background difference algorithm to detect whether there is a moving target in the starting frame and the ending frame of a video clip, and determines whether the movement of the moving target is violent. If there is a moving target, it means that the contents of each frame in the current video clip are different from each other. At this time, the video clip is a dynamic video clip and should be transcoded into a dynamic data body. Correspondingly, if a moving target cannot be detected from the starting frame and the ending frame of the video clip, it means that the video clip may be a static picture throughout, and the content of each frame is the same. At this time, the video clip is a static video clip and should be transcoded into a static data body.

[0062] Specifically, the background difference method is an effective moving object detection algorithm, which has a good detection effect in scenes where the background image remains static, such as surveillance videos. The basic idea of ​​the algorithm is to use the parameter model of the background to approximate the pixel value of the background image, and to perform a differential comparison between the current frame and the background image to detect the moving area, where the pixel area with a larger difference is considered to be the moving area, and the pixel area with a smaller difference is considered to be the background area. The background difference method must have a background image, and the background image must be updated in real time as the lighting or external environment changes. Therefore, the key to the background difference method is background modeling and its update.

[0063] In practical applications, this embodiment extracts pixels of the moving target part from the starting frame and the ending frame in turn through the background difference algorithm, and then compares the starting frame and the ending frame to determine whether the movement amplitude of the moving target in the duration corresponding to the video clip exceeds a preset threshold. At this stage, the threshold of this embodiment is set relatively high, that is, only when the difference between the previous and next frames of the video clip is relatively significant will it be considered that there is a moving target with a relatively violent movement amplitude in the video clip. If the target in the video screen moves violently, the difference between the frames of the video clip is relatively large, and it should be transcoded into a dynamic data body. On the contrary, it means that the target remains still or moves relatively slowly, and the difference between the frames of the video clip is relatively small, and it should be transcoded into a static data body.

[0064] Compared with dynamic data, the transcoding process of static data is relatively simple. In the new data format, only the image of the start frame and the timestamp information of the start frame and the end frame need to be retained. Figure 2 As shown, the data format of the static data body obtained after transcoding in this embodiment is as follows:

[0065] {data: {p0}, timestamp: {dy_x1, dy_x2}}

[0066] Among them, data represents the data segment, timestamp represents the marker segment; p0 represents the image data recorded in the data segment; dy_x1 and dy_x2 represent the timestamps corresponding to the start frame and end frame of the video segment corresponding to the static data body, respectively.

[0067] When the data format is decoded, the display terminal directly displays the image data p0 in the data segment from the dy_x1 and dy_x2 moments. During this process, the screen does not need to be refreshed, indicating that there is no new content in the monitoring screen of the entire process.

[0068] S3: The previous article introduced that the video clips storing moving targets in the content should be transcoded into dynamic data bodies. In the previous steps, a dynamic tag was added to the video clips that should be transcoded into dynamic data bodies. In this step, the video clips with dynamic tags need to be transcoded into dynamic data bodies in a specified format.

[0069] It should be noted that: theoretically, the dynamic data body should save the pictures of each frame in the original video clip to avoid information loss. However, in actual situations, the video clip time that should be transcoded into the dynamic data body is also divided into many different situations, for example, Figure 3 Four different types of pictures corresponding to dynamic data bodies are displayed respectively. Among them, the first type means that in the entire video clip, the moving target always keeps moving; at this time, the picture contents of all frames of the video clip are different. The second type means that in the entire video clip, the moving target keeps moving in the early stage and keeps still in the later stage; at this time, there are differences in the pictures of the early frames in the video clip, and the pictures of the later frames are the same. The third type means that in the entire video clip, the moving target keeps still in the early stage and keeps moving in the later stage; at this time, the pictures of the early frames in the video clip are the same, and there are differences in the pictures of the later frames. The fourth type means that in the entire video clip, the moving target keeps still in the early and late stages, and keeps moving in one or some periods in the middle; at this time, the pictures of the early and late frames in the video clip are the same, and there are differences in the pictures of the corresponding periods in the middle.

[0070] Therefore, in order to save storage space to the greatest extent and improve data compression rate, the dynamic data body should only retain the frames of the period when the target remains in motion, and only one frame (such as the start frame) is required for the period when the target remains stationary. Figure 4 As shown in FIG. 1 , the process of generating a dynamic data volume using a video clip with dynamic labels is as follows:

[0071] A low-threshold background subtraction algorithm is used to compare and analyze any two adjacent frames in a video clip with dynamic labels, and the following decisions are made:

[0072] (1) If there is a change between any two frames, the image data of the previous frame is retained as the image data of the data segment, and the time information of the previous frame is recorded as the dynamic timestamp of the marker segment, thereby obtaining a data element.

[0073] (2) If there is no change between any two frames, continue to compare the next frame until all frames in the video clip are identified.

[0074] All data elements obtained from each video segment with dynamic tags are arranged according to timestamps to form a dynamic data body of the current video segment.

[0075] Background difference algorithm is adopted in both this step and step S2, but there are differences in the purpose of the two. In step S2, the background difference algorithm is used to detect whether there is a moving target in the video screen through the starting frame and the ending frame. In step S3, the background difference algorithm is used to detect the moving target in the screen in which time periods remain moving and which time periods remain static through frame-by-frame data. From a functional point of view, the latter should be more sophisticated in identifying the motion state of the target, so in step S3, this embodiment sets a lower threshold for motion detection, which makes the target only need to have a small displacement in the screen in two adjacent frames, that is, it is considered to be in motion rather than static state.

[0076] In addition, in the above two steps, when the background difference algorithm is running, the modeling and simulation of the background becomes more difficult due to the complexity and unpredictability of the scene, as well as the existence of various environmental interferences and noises, such as sudden changes in illumination, fluctuations of some objects in the actual background image, camera jitter, and the impact of moving objects entering and exiting the scene on the original scene. Therefore, in this embodiment, at the beginning of the algorithm running, the first frame image is selected as the initial modeling background, and then the algorithm uses the image data recorded in the data segment of any previous static data body as the modeling background of the subsequent video segment with dynamic labels.

[0077] S4: extracting image data from the data segment of each data element in the dynamic data body, inputting it into a target recognition algorithm, and adding an attribute segment for recording relevant information in each data element according to the recognition result.

[0078] In step S3, the scheme of this embodiment performs moving target detection on the video clip containing the moving target, and then obtains a series of metadata. Each metadata screen includes a moving target. Therefore, this embodiment further performs target recognition on the images in each metadata. The target recognition algorithm that can be used in this embodiment can use methods such as R-CNN, Fast R-CNN, Faster R-CNN or YOLO (including YOLOv3~YOLOv7). The target recognition algorithm can identify the types of all moving targets included in the screen and their corresponding scene information. For example, when pedestrians, animals and vehicles appear in the screen, the target recognition algorithm can identify the types of these targets. In addition, the target recognition algorithm can even identify and record the current natural phenomena in the screen, such as rain, snow, lightning, etc. This embodiment adds the detected information as attribute data to the dynamic data body, thereby allowing the user to query the video content based on the text information, and realize the retrieval of the video content without playing the video stream data.

[0079] On this basis, if Figure 2As shown, the data format of the dynamic data body in this embodiment is as follows:

[0080] {data: {p1}, timestamp: {stc_x1}, type: {c1},…, data: {pn}, timestamp: {stc_xn},

[0081] type: {cn}}

[0082] Among them, data represents the data segment, timestamp represents the marker segment; type represents the attribute segment; p1~pn represent the image data recorded in the data segments of the 1st to nth data elements respectively; stc_x 1~stc_x n represent the timestamps corresponding to the start frames of the 1st to nth data elements respectively; c1~cn represent the related information recorded in the attribute segments of the 1st to nth data elements respectively.

[0083] When the data is decoded, the display terminal dynamically refreshes the screen content according to the image data of the data segment and the timestamp information of the marker segment. When the target in the video screen keeps moving, the refresh rate of the screen is high, and when the target in the video screen remains static, the refresh rate of the screen will decrease.

[0084] The content of the attribute segment in this embodiment is not used for display, but only for supporting users to search for video content. When users search for video content by keywords, the computer will match the keywords entered by the user with the content of the attribute segment of the dynamic data weight, and return the time when the target appears in the video screen (i.e. the timestamp information of the corresponding mark segment) to the user based on the matching results. Users can jump to the corresponding time node to watch the corresponding video screen based on the query results.

[0085] S5: Temporarily store the acquired static data body and dynamic data body locally, and upload the temporarily stored data to the data center when the amount of stored data exceeds a preset capacity upper limit.

[0086] Although the scheme of this embodiment can achieve a higher video data compression rate than the prior art, when the video acquisition equipment such as the surveillance camera runs for a long time, the backend of this embodiment will still generate a large amount of storage data. In view of this situation, this embodiment chooses to temporarily store the relevant data locally, and regularly uploads the temporarily stored data to the cloud or backend server for permanent storage.

[0087] It should be noted that: although the original video stream data collected by the camera has a large amount of data, it also contains the richest amount of information. Although the new video data format composed of dynamic data bodies and static data bodies provided in this embodiment can achieve data compression, it will inevitably cause information loss when comparing and selectively storing each frame of the image using the background difference method. Therefore, in actual application scenarios, the terminal device can flexibly choose between the two data storage methods according to needs. Customers or devices with higher requirements for data security are suitable for choosing traditional video data storage methods, while customers or devices with higher requirements for storage space optimization performance are suitable for choosing the new video data storage method provided in this embodiment. Of course, the relevant equipment can also flexibly switch between the two storage methods in different time periods or scenes, or use both storage methods at the same time, and use the two data contents to respond to different service needs of users.

[0088] Example 2

[0089] Based on the solution of Example 1, this embodiment further provides a video encoder for monitoring video data, which is used to optimize the data structure of the video stream data collected by the camera using the monitoring video data storage method optimized by the data structure of Example 1. Figure 5 As shown, the video encoder provided by this embodiment includes: a data acquisition unit, a first encoding unit, a second encoding unit, and an attribute generation unit.

[0090] The data acquisition unit is used to obtain the video stream data collected by the front-end camera in real time, and generate video clips of specified length according to preset time intervals.

[0091] The first encoding unit is used to compare the start frame and the end frame of each video segment through a background difference algorithm with a high threshold value to see whether they have changed: if there is a change between the two, a dynamic tag is added to the video segment. If there is no change between the two, the start frame image is recorded as the image data of the data segment, and the time information from the start frame to the end frame is recorded as the static timestamp of the mark segment; thus, a static data body is obtained.

[0092] The second encoding unit is used to obtain the video segment with dynamic tags output by the first encoding unit. Then, a background difference algorithm with a low threshold is used to compare whether any two adjacent frames in the video segment with dynamic tags have changed: if there is a change between any two frames, the image data of the previous frame is retained as the image data of the data segment, and the time information of the previous frame is recorded as the dynamic timestamp of the marked segment, thereby obtaining a data element. If there is no change between any two frames, the next frame is compared until all frames in the video segment are identified. Finally, all data elements obtained from the video segment are arranged according to the timestamp to form the dynamic data body of the current video segment.

[0093] The attribute generation unit is used to extract the image data of the data segment in each data element of the dynamic data body, input it into a target recognition algorithm, and add an attribute segment for recording relevant information between the corresponding data element according to the recognition result.

[0094] The solution provided in this embodiment realizes the strategy of optimizing the storage space of video stream data, which is essentially to compress the video data in time. Therefore, in the solution further optimized in this embodiment, Figure 6 As shown, the video encoder of the surveillance video data may further include a recompression unit. The recompression unit is used to compress the image data recorded in the data segments of the static data body and the dynamic data body by using a spatial compression or DCT method, so as to further compress the new video data format composed of the dynamic data body and the static data body.

[0095] Example 3

[0096] Based on the above embodiments, this embodiment further provides a video monitoring system. Figure 7 As shown, it includes a local device and a remote data center; the data center is used to permanently store the video data uploaded by the local device and respond to query requests for video content from authorized users.

[0097] The local devices in the video monitoring system include: a camera, a data processing module and a storage module. The camera is used to collect the original video stream data in the target area.

[0098] The data processing module runs a video encoder for monitoring video data as in Example 2. The video encoder is used to obtain the original video stream data collected by the camera, and optimize the data structure thereof, thereby obtaining new video stream data consisting of a static data body and a dynamic data body.

[0099] The storage module includes a memory and a storage management unit. The storage management unit is used to store original video stream data and / or new video stream data in the memory, and when the remaining capacity of the memory is lower than a preset capacity lower limit, upload the data stored in the memory to a background data center.

[0100] The data processing module and storage module in the local device in this embodiment essentially constitute a microcomputer system capable of executing computer programs. In practical applications, this part can be implemented through embedded devices, and can also be adapted to various general-purpose computing devices, such as laptops, tablet computers, desktops, edge servers, and so on.

[0101] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A monitoring video data storage method with optimized data structure, characterized in that: It includes the following steps: S1: Segment the video stream data collected by the front end into several video segments of specified length according to a preset time interval; S2: A high-threshold background difference algorithm is used to compare and analyze the start and end frames of each video clip and make the following decisions: (1) If there is a moving target in the video clip, add a dynamic label to each video clip; (2) If there is no moving target in the video clip, the start frame image is recorded as the image data of the data segment, and the time information from the start frame to the end frame is recorded as the static timestamp of the marker segment, thereby obtaining a static data body; S3: Compare and analyze any two adjacent frames in the video clip with dynamic labels through a background difference algorithm with a low threshold, and make the following decisions: (1) If there is a change between any two frames, the image data of the previous frame is retained as the image data of the data segment, and the time information of the previous frame is recorded as the dynamic timestamp of the marker segment, thereby obtaining a data element; (2) If there is no change between any two frames, continue to compare the next frame until all frames in the video clip are identified; All data elements obtained from each video segment with dynamic tags are arranged according to timestamps to form a dynamic data body of the current video segment; S4: extracting image data from the data segment of each data element in the dynamic data body, inputting the image data into a target recognition algorithm, and adding an attribute segment for recording relevant information in each data element according to the recognition result; S5: Temporarily store the acquired static data body and dynamic data body locally, and upload the temporarily stored data to the data center when the amount of stored data exceeds a preset capacity upper limit.

2. The monitoring video data storage method with optimized data structure as claimed in claim 1, characterized in that: The data format of the static data body is as follows: {data: {p0}, timestamp: {dy_x1, dy_x2}} Among them, data represents the data segment, timestamp represents the marker segment; p0 represents the image data recorded in the data segment; dy_x1 and dy_x2 represent the timestamps corresponding to the start frame and end frame of the video segment corresponding to the static data body, respectively.

3. The monitoring video data storage method with optimized data structure as claimed in claim 1, characterized in that: The data format of the dynamic data body is as follows: {data: {p1}, timestamp: {stc_x1}, type: {c1},…, data: {pn}, timestamp: {stc_xn}, type: {cn}} Among them, data represents the data segment, timestamp represents the marker segment; type represents the attribute segment; p1~pn represent the image data recorded in the data segments of the 1st to nth data elements respectively; stc_x 1~stc_x n represent the timestamps corresponding to the start frames of the 1st to nth data elements respectively; c1~cn represent the related information recorded in the attribute segments of the 1st to nth data elements respectively.

4. The monitoring video data storage method with optimized data structure as claimed in claim 1, characterized in that: In step S1, the preset time interval does not exceed 5 seconds.

5. The monitoring video data storage method with optimized data structure as claimed in claim 1, characterized in that: In step S2 and step S3, the first frame image is used as the initial modeling background, and the image data recorded in the data segment of any preceding static data volume is used as the modeling background of the subsequent video segment with dynamic labels.

6. The monitoring video data storage method with optimized data structure as claimed in claim 1, characterized in that: In step S4, the target recognition algorithm adopts a method based on R-CNN, Fast R-CNN, Faster R-CNN or YOLO.

7. The monitoring video data storage method with optimized data structure as claimed in claim 1, characterized in that: In step S4, the relevant information recorded in the attribute segment of the dynamic data body includes the types of all moving objects contained in the identified image and their corresponding scene information.

8. A video encoder for surveillance video data, characterized in that: It is used to optimize the data structure of the video stream data collected by the camera by using the monitoring video data storage method with data structure optimization as described in any one of claims 1 to 6; the video encoder includes: The data acquisition unit is used to acquire the video stream data collected by the front-end camera in real time and generate video clips of a specified length at preset time intervals; The first encoding unit is used to compare whether the start frame and the end frame of each video segment have changed by using a background difference algorithm with a high threshold value: if there is a change between the two, a dynamic label is added to the video segment; if there is no change between the two, the start frame image is recorded as the image data of the data segment, and the time information from the start frame to the end frame is recorded as the static timestamp of the mark segment; thereby obtaining a static data body; A second encoding unit is used to obtain the video segment with dynamic tags output by the first encoding unit; then compare whether any two adjacent frames in the video segment with dynamic tags have changed by using a background difference algorithm with a low threshold: if there is a change between any two frames, the image data of the previous frame is retained as the image data of the data segment, and the time information of the previous frame is recorded as the dynamic timestamp of the marking segment, thereby obtaining a data element; if there is no change between any two frames, the next frame is compared until all frames in the video segment are identified; finally, all data elements obtained from the video segment are arranged according to the timestamp to form a dynamic data body of the current video segment; The attribute generation unit is used to extract the image data of the data segment in each data element of the dynamic data body, input it into a target recognition algorithm, and add an attribute segment for recording relevant information between the corresponding data element according to the recognition result.

9. The video encoder for monitoring video data according to claim 8, characterized in that: It also includes a recompression unit; the recompression unit is used to compress the image data recorded in the data segments in the static data body and the dynamic data body by adopting the spatial compression or DCT method.

10. A video surveillance system, characterized in that: It includes a local device and a remote data center; the data center is used for the local device to permanently store the uploaded video data and respond to the query request of the authorized user for the video content; The local device includes: A camera is used to collect raw video stream data in a target area; A data processing module, in which a video encoder for monitoring video data as claimed in claim 8 or 9 is running; the video encoder is used to obtain the original video stream data collected by the camera, and optimize the data structure thereof, thereby obtaining new video stream data composed of a static data body and a dynamic data body; A storage module includes a memory and a storage management unit, wherein the storage management unit is used to store the original video stream data and / or the new video stream data in the memory, and when the remaining capacity of the memory is lower than a preset capacity lower limit, upload the data stored in the memory to a background data center.

Images