Video data additional recording method, video monitoring system and readable storage medium

Abstract

The invention provides a video data additional recording method, a video monitoring system and a readable storage medium, and the video data additional recording method is applied to a back-end server of the video monitoring system, and comprises the steps: carrying out the integrity detection of an obtained media stream, generating a corresponding first additional recording task according to the integrity detection result, and storing the first additional recording task in the video monitoring system; and executing the first additional recording task to realize complete acquisition of the media stream by the back-end server. According to the technical scheme provided by the invention, the integrity of the media stream can be detected in real time in the process of synchronously storing the media stream at the rear end, the transmission damage condition of the media stream can be acquired, and the corresponding first additional recording task is generated according to the transmission damage condition, so that the complete acquisition of the media stream by a rear-end server is realized.

Description

Technical Field

[0001] The present disclosure relates to the technical field of video surveillance storage, and in particular to a video data supplementary recording method and a video surveillance system. Background Art

[0002] In video surveillance applications, the video images collected by the monitoring front-end device are usually transmitted to the back-end video surveillance platform for centralized video storage. In some important scenarios, the front-end device will also be equipped with a storage card for local backup storage, or a network video recorder (NVR) will be deployed in the edge area close to the front-end device to connect to the camera and record nearby. The front-end device will then be connected to the back-end video surveillance platform through the network after aggregation through the network video recorder, and the video will be centrally stored on the back-end server.

[0003] Since the network from the front-end device or network video recorder to the back-end video surveillance platform is usually complex, if the video image is transmitted and stored, such as network congestion leading to packet loss or network disconnection, the back-end platform's video recording will be incomplete or missing. In order to deal with these problems and ensure the integrity of the back-end platform's video recording, it is necessary to supplement the local video recording stored in the front-end device or the video recording cached on the network video recorder to the storage space of the back-end server through supplementary recording and other means.

[0004] Traditional solutions can only deal with the situation where the recording is missing or discontinuous due to network disconnection, but cannot solve the problem of continuous but incomplete recording. Summary of the invention

[0005] In response to the problems raised in the background technology, the present invention provides a video data re-recording method, a video surveillance system and a readable storage medium, which can perform integrity detection and generate corresponding re-recording tasks for scenarios where media stream transmission is damaged, and provide an optimized re-recording solution when re-recording is required.

[0006] Specifically, the first aspect of the present disclosure provides a video data supplementary recording method, which is applied to a backend server in a video surveillance system, wherein the backend server responds to a recording request, obtains a media stream transmitted by a front-end device and stores the media stream;

[0007] Specifically, the video data supplementary recording method includes the following steps:

[0008] Performing integrity detection on the acquired media stream to obtain the transmission quality status of the media stream, where the transmission quality status includes the transmission damage status of the media stream;

[0009] Generate a corresponding first supplementary recording task according to the transmission quality status and the video data supplementary recording plan, where the first supplementary recording task covers a corresponding time period in which the transmission is damaged in the media stream;

[0010] The first supplementary recording task is executed to enable the back-end server to fully acquire the media stream.

[0011] In a possible implementation of the first aspect, the process of storing the acquired media stream includes the following steps:

[0012] Encapsulate the acquired media stream and segment it according to a fixed duration to form multiple segmented data blocks and store them in the back-end server, and generate a video record corresponding to each segmented data block;

[0013] The video recording includes at least one or a combination of multiple items of the front-end device information, segment time information, storage location information and transmission quality status corresponding to each segment data block.

[0014] In a possible implementation of the first aspect, in a process of generating a corresponding first supplementary recording task according to a transmission quality state and a video data supplementary recording plan, the following steps are included:

[0015] Scan all video records corresponding to the media stream to be detected to obtain the damaged video duration corresponding to each video record, where the damaged video duration includes the sum of the damaged durations of each image group in the segmented media stream corresponding to the video record;

[0016] Based on the video data re-recording plan, when the duration of the damaged video is greater than a first preset threshold, a first re-recording task is generated. The first re-recording task includes re-acquiring historical media streams corresponding to all segmented data blocks with damaged frames from the front-end device.

[0017] In a possible implementation of the first aspect, the process of obtaining the damaged duration of each image group in the segmented media stream includes the following steps:

[0018] Based on the video recording, obtain the distribution of damaged frames in each image group;

[0019] The duration from the first damaged frame to the next key frame in each image group is taken as the damaged duration of the image group.

[0020] In a possible implementation of the first aspect above, the integrity detection includes performing packet loss detection on the media stream to be detected and / or performing tamper-proof verification on built-in tamper-proof information of the media stream to be detected;

[0021] Obtaining a distribution of damaged frames of the to-be-detected media stream based on the packet loss detection and / or the anti-tampering verification;

[0022] Obtaining the number of packet losses and / or the packet loss rate of the media stream to be detected based on the packet loss detection;

[0023] In the process of generating the first supplementary recording task, based on the video data supplementary recording plan, when the number of packet losses is greater than a second preset threshold and / or the packet loss rate is greater than a third preset threshold, the corresponding first supplementary recording task is generated.

[0024] In a possible implementation of the first aspect, the video data supplementary recording method further includes:

[0025] Scan all video records corresponding to the media stream to be detected to obtain the start timestamp and the end timestamp of all video records;

[0026] According to the start timestamp and the end timestamp, determine whether the media stream to be detected is continuous: if not, obtain the missing time period corresponding to the media stream to be detected;

[0027] Based on the video data re-recording plan, when the duration of the missing period is greater than the fourth preset threshold, a second re-recording task is generated, and the second re-recording task includes re-acquiring historical media streams corresponding to all video records associated with the missing period from the front-end device.

[0028] In a possible implementation of the first aspect, each first supplementary recording task includes front-end device information of the supplementary recording media stream, supplementary recording start and end time information of the supplementary recording media stream, and supplementary recording execution priority;

[0029] The priority of re-recording execution is positively correlated with the duration of damaged video in the media stream;

[0030] The first supplementary recording task is executed according to the preset priority rules, including:

[0031] Obtaining a first supplementary recording task to be executed to generate a corresponding supplementary recording execution plan;

[0032] Obtain the network transmission environment between the front-end device and the back-end server;

[0033] When the network transmission environment does not support the supplementary recording execution plan, the first supplementary recording task is executed in sequence according to the supplementary recording execution priority; and / or

[0034] Adjust the number of concurrently executed first supplementary recording tasks.

[0035] In a possible implementation of the first aspect, each segmented media stream corresponding to each segmented data block uses a key frame as a start frame;

[0036] Executing the first supplementary recording task includes the following steps:

[0037] Maintain clock synchronization between front-end devices and back-end servers;

[0038] According to the supplementary recording start and end time information of the supplementary recording media stream, a preset margin is added to obtain the excess deadline time corresponding to the supplementary recording media stream, and the preset margin is less than the duration of a single image group in the supplementary recording media stream;

[0039] Based on the supplementary recording start and end time information and the overrun deadline, the historical media stream is re-obtained from the front-end device, and the last key frame in the historical media stream is used as the supplementary recording deadline;

[0040] The historical media stream is trimmed according to the supplementary recording cutoff position to obtain the supplementary recording media stream.

[0041] A second aspect of the present disclosure provides a video monitoring system, which may specifically include:

[0042] Front-end equipment;

[0043] The back-end server is used to respond to the recording request, obtain the media stream transmitted by the front-end device and store it;

[0044] The back-end server is also used to adopt the video data supplementary recording method provided in the first aspect to achieve complete acquisition of the media stream.

[0045] A third aspect of the present disclosure provides a computer-readable storage medium for storing a program, wherein when the program is executed by a processor, the steps of the video data supplementary recording method provided in the first aspect are implemented.

[0046] Compared with the background technology, the present invention has the following beneficial effects:

[0047] The technical solution provided by the present disclosure can detect the integrity of the media stream in real time during the process of synchronously storing the media stream at the back end, can obtain the transmission damage of the media stream, and generate a corresponding first re-recording task according to the transmission damage, thereby realizing the back-end server's complete acquisition of the media stream; by taking the transmission damage of the video media stream as the detection object and for generating the re-recording task, the video re-recording situation in which the video stream transmission is incomplete although the video recording is continuous is solved.

[0048] Specifically, the technical solution provided by the present invention introduces multi-dimensional evaluation indicators such as media stream loss, packet loss rate and / or number of packet losses, and video damage duration during the detection process. It can more accurately and intuitively reflect the integrity of media stream acquisition from the two aspects of damage and loss, and is closer to the user's experience and perception. It can realize automatic detection and re-recording of media stream transmission for common application scenarios such as network congestion packet loss, video transmission loss, and video transmission damage that affect the integrity of the video, and can adjust the setting and execution of the re-recording task according to changes in the network transmission environment, which has popularizable value. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Other features, objects and advantages of the present disclosure will become more apparent from a reading of the detailed description of non-limiting embodiments made with reference to the following accompanying drawings.

[0050] Figure 1 According to an embodiment of the present disclosure, a schematic flow chart of a method for supplementary recording of video data is shown.

[0051] Figure 2 According to an embodiment of the present disclosure, a schematic diagram of the structure of a computer-readable storage medium is shown. DETAILED DESCRIPTION

[0052] The following describes the embodiments of the present disclosure through specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in the present disclosure. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in the present disclosure can also be modified or changed in various ways according to different viewpoints and application systems without departing from the spirit of the present disclosure. It should be noted that the embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0053] The following is a detailed description of the embodiments of the present disclosure with reference to the accompanying drawings so that those skilled in the art can easily implement the present disclosure. The present disclosure can be embodied in many different forms and is not limited to the embodiments described herein.

[0054] In the representations of the present disclosure, the reference terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" and the like mean that the specific features, structures, materials, or characteristics represented in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. Moreover, the specific features, structures, materials, or characteristics represented may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples represented in the present disclosure and the features of different embodiments or examples, unless they are mutually contradictory.

[0055] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the representation of the present disclosure, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0056] In order to clearly describe the present disclosure, components not related to the description are omitted, and the same reference numerals are given to the same or similar components throughout the specification.

[0057] Throughout the specification, when a device is said to be "connected" to another device, this includes not only the case of "direct connection" but also the case of "indirect connection" by placing other elements therebetween. In addition, when a device is said to "include" a certain component, unless otherwise stated, it does not exclude other components, but means that other components may be included.

[0058] When a device is said to be "on" another device, it may be directly on the other device, but there may also be other devices between it. In contrast, when a device is said to be "directly" on another device, there are no other devices between it.

[0059] Although the terms first, second, etc. are used to represent various elements in some examples, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, the first interface and the second interface, etc. are represented. Moreover, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising" and "including" indicate the existence of features, steps, operations, elements, components, projects, kinds, and / or groups, but do not exclude the existence, occurrence or addition of one or more other features, steps, operations, elements, components, projects, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Therefore, "A, B or C" or "A, B and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C". Only when the combination of elements, functions, steps or operations is inherently mutually exclusive in some way, will there be an exception to this definition.

[0060] The technical terms used herein are only used to refer to specific embodiments and are not intended to limit the present disclosure. The singular form used herein also includes the plural form unless the sentence clearly indicates the contrary meaning. The meaning of "including" used in the specification is to specify specific characteristics, regions, integers, steps, operations, elements and / or components, and does not exclude the existence or addition of other characteristics, regions, integers, steps, operations, elements and / or components.

[0061] Although not defined differently, all terms, including technical and scientific terms used herein, have the same meaning as those generally understood by those skilled in the art to which the present disclosure belongs. Terms defined in commonly used dictionaries are additionally interpreted as having meanings consistent with the content of relevant technical literature and current disclosures, and unless defined, shall not be overly interpreted as ideal or very formal meanings.

[0062] In order to overcome the problems raised in the background technology, the present disclosure provides a video data re-recording method and a video surveillance system, which can perform integrity detection and generate corresponding re-recording tasks for scenes where media stream transmission is damaged, and provide an optimized re-recording solution when re-recording is required. Specifically, the video data re-recording method provided by the present disclosure is applied to a video surveillance system including a front-end device and a back-end server, wherein the front-end device includes a recording device and a network video recorder, which has the ability to record locally and can respond to real-time media stream requests and historical media stream requests initiated by the back-end server; the back-end server responds to the recording request, obtains the media stream transmitted by the front-end device and stores it, which will not be described in detail below.

[0063] In some embodiments of the present disclosure, Figure 1 FIG. 1 shows a flow chart of a method for supplementing video data recording. Figure 1 As shown, the supplementary recording method may specifically include the following steps:

[0064] Step 100: Perform integrity detection on the acquired media stream to obtain the transmission quality status of the media stream. The transmission quality status may include the transmission damage status corresponding to the media stream. It is understandable that in the above step 100, the integrity detection is mainly used to detect in real time whether there is packet loss in the media stream or tampering of anti-tampering information in the process of receiving the code stream corresponding to the media stream, and the media stream damage caused by data transmission. The specific acquisition of the transmission damage status will be explained later, and will not be repeated here.

[0065] Step 200: Generate a corresponding re-recording task according to the transmission damage and the preset video data re-recording plan. The video data re-recording plan can also be called a re-recording job, which scans the integrity detection result of the media stream and determines whether to generate the corresponding first re-recording task according to whether the transmission damage matches the preset damage index, wherein the preset damage index can be set by the user according to the actual situation. It can be understood that a video surveillance system usually includes multiple front-end devices, and the user can select one or more front-end devices as the focus of attention according to actual needs, and configure the re-recording plan for the media streams provided by these front-end devices. The parameters included in the re-recording plan may specifically include the media streams transmitted by which front-end devices need to be scanned, the damage threshold conditions for determining whether to generate the first re-recording task, the execution time of generating the re-recording task, etc., which will not be elaborated here.

[0066] In the above step 200, the first supplementary recording task corresponds to the to-be-recorded scene corresponding to the damaged situation of the video stream during the transmission process, which is configured to cover the corresponding time period in which the transmission is damaged in the media stream, thereby ensuring that the supplementary recording task can eliminate the impact of the damaged transmission of the media stream. Specifically, the implementation of the damaged transmission detection will be described later.

[0067] Step 300: Execute the supplementary recording task to achieve complete acquisition of the media stream by the back-end server. It is understandable that, during the execution of the above-mentioned first supplementary recording task, in order to avoid network congestion caused by the simultaneous execution of multiple first supplementary recording tasks, and to avoid the execution of the first supplementary recording task affecting the normal storage of other subsequent video streams, the specific implementation order and implementation time of the first supplementary recording task can be further limited according to the preset priority rules, so as to avoid new transmission damage or transmission loss in the supplementary recording link to the greatest extent. The specific execution of the first supplementary recording task will be explained later.

[0068] It can be understood that through the above steps 100 to 300, the problem of incomplete media stream caused by damaged media stream transmission, which is difficult to solve in the prior art, can be solved: by detecting the integrity of the media stream in real time during the process of synchronously storing the media stream at the back end, the transmission damage of the media stream can be obtained, and then the corresponding first supplementary recording task can be configured according to the transmission damage, so as to achieve the back end server to obtain the complete media stream. The specific implementation of the above steps 100 to 300 will be further explained below.

[0069] In the above embodiment, specifically, in the process of storing the acquired media stream by the back-end server, a segmented storage method can be adopted, that is, after encapsulating the acquired media stream, it is segmented according to a preset fixed time length, and the segmented encapsulated media stream is stored in the storage space divided by the back-end server in the form of segmented files or segmented data blocks as a storage video corresponding to the video surveillance. Further, in the above process of segmented storage, a video record corresponding to the current media stream can be generated synchronously, and the video record here is used to record and save the metadata information corresponding to each segmented data block, which can specifically include the front-end device information corresponding to the segmented data block, such as the identity identification code of the front-end device, etc.; segmentation time information, such as the start timestamp and end timestamp of the segmented media stream corresponding to the segmented data block, etc.; storage location information, such as the storage path of the segmented data block in the first storage space, etc.; integrity detection results, such as whether the segmented media stream corresponding to the segmented data block has transmission damage, etc. It is understandable that, in the process of storing the obtained media stream by the back-end server, the detection result of the integrity detection of the media stream will be synchronized to the video record, so that the subsequent video data supplementary recording plan can scan the video record to generate the corresponding supplementary recording task. Those skilled in the art can set the specific metadata information contained in the video record according to actual needs, which is not limited here. In order to be able to quickly retrieve and obtain the corresponding segmented video recordings of each segmented data block through the video record, the above-mentioned video records can be centrally stored to form a video record table. In a specific implementation of the present disclosure, the above-mentioned video records can be stored in the form of a database, which is not limited here. Accordingly, in the process of generating the corresponding first supplementary recording task based on the preset video data supplementary recording plan, the video data supplementary recording plan can be set to scan and traverse all the video records corresponding to the media stream to be detected, and determine whether there is a transmission damage according to the integrity detection result in the video record and configure the corresponding first supplementary recording task.

[0070] In the above embodiment, in the specific implementation of the aforementioned step 100, the integrity detection may include performing packet loss detection on the media stream to be detected to obtain the number of packet losses and / or the packet loss rate of the media stream to be detected. In an optional specific implementation, the video data supplementary recording plan may be configured to generate a corresponding damaged supplementary recording task when the number of packet losses of the media stream to be detected is greater than the second preset threshold and / or the packet loss rate is greater than the third preset threshold. Further, the integrity detection may include performing anti-tampering verification on the embedded anti-tampering information of the media stream to be detected to achieve a more comprehensive integrity detection of the media stream. It can be understood that according to the packet loss detection and anti-tampering verification in the integrity detection, the distribution of damaged frames in the media stream and the number of packet losses / packet loss rate of the media stream can be obtained, wherein the number of packet losses and / or the packet loss rate can be directly obtained according to the packet loss detection, and the damaged frame can be obtained according to the packet loss detection and / or the anti-tampering verification. If a certain image frame has packet loss and / or fails the anti-tampering verification, it will be marked as a damaged frame and recorded in the video recording. These indicators obtained based on the integrity detection can comprehensively reflect the transmission damage of the media stream.

[0071] In the above embodiment, further, in the process of configuring the supplementary recording task, the present disclosure introduces a damaged duration indicator in the video data supplementary recording plan, which is used to accurately reflect the damage of the media stream in a longer period of time. Specifically, the damaged video duration corresponding to each video record can be obtained by scanning all the video records corresponding to the media stream to be detected, wherein the damaged video duration includes the sum of the damaged durations of each group of pictures (Group of Pictures, GOP) in the segmented media stream corresponding to the video record. It can be understood that the picture group is a group of pictures in the segmented data block used to assist in random access to continuous pictures, wherein the first picture frame of each picture group is a key frame, which can ensure that a single picture group can be independently encapsulated and decoded without referring to other pictures. Specifically, in the process of obtaining the damaged duration corresponding to each picture group, the distribution of damaged frames in each picture group can be obtained based on the video record, and the duration between the first damaged frame and the next key frame in each picture group is used as the damaged duration of the picture group. The damaged frame can be determined by the integrity detection process in the aforementioned step 100 according to whether the video frame has packet loss or tampering verification failure, which is not limited here.

[0072] In the above embodiment, based on the video data re-recording plan, when the duration of the damaged video is greater than the first preset threshold, it means that the damaged duration corresponding to the segmented media stream has reached the set damaged threshold, and a corresponding re-recording task needs to be generated. The re-recording task can be configured to re-acquire the historical media stream corresponding to all segmented data blocks with damaged frames from the corresponding front-end device.

[0073] It is understandable that in the aforementioned process of transmission damage detection based on the number of packet losses and / or the packet loss rate, since the packet loss may occur in the form of sudden concentrated packet loss or occasional random packet loss, it is difficult to accurately reflect the extent of damage to the video over a longer period of time through the number of packet losses and / or the packet loss rate indicators. The present disclosure introduces a damage duration indicator that can more comprehensively reflect the extent of damage to the video over a longer period of time, and is closer to the user experience quality.

[0074] It can be understood that in the process of evaluating the transmission damage of the media stream in the aforementioned embodiments of the present disclosure, evaluation indicators such as video damage duration, number of packet losses and / or packet loss rate are introduced, which can evaluate the transmission damage of the media stream from multiple aspects, and the damage obtained by evaluation is consistent with the actual experience of the user, and can realize accurate and intuitive evaluation of the integrity of the media stream.

[0075] However, considering that the detection of transmission damage usually does not include the detection of missing media streams, this is because the missing media streams may be caused by transmission factors such as network interruption, as well as storage failures of the backend server itself. In order to cover the above-mentioned video recording missing scenarios, the continuity of the media stream obtained by the background server is detected for video integrity. In some embodiments of the present disclosure, the video records can be scanned based on the video data supplementary recording plan to obtain the start timestamp and end timestamp corresponding to all the video data of the media stream to be detected; further, based on the start timestamp and end timestamp, it can be determined whether the media stream to be detected is continuous: if not, the missing period corresponding to the media stream to be detected is obtained. When the missing period reaches the specified missing threshold, such as the duration of the missing period is greater than the fourth preset threshold, a second supplementary recording task is generated. It can be understood that the second supplementary recording task corresponds to the situation where the media stream is missing during the transmission and / or storage process, including re-obtaining the historical media stream corresponding to all the segmented data blocks associated with the missing period from the front-end device. That is to say, in the above embodiment, the video recordings can also be scanned based on the video data re-recording plan. Different from the transmission damage obtained by integrity detection in the previous embodiment, the continuity of the media stream is verified by manually set missing threshold indicators, so that the scene of missing media stream can be identified and the corresponding second re-recording task can be generated. In some specific implementations of the embodiments of the present disclosure, only the first re-recording task can be executed according to actual needs, that is, only the scene where the media stream received by the back-end server is damaged can be re-recorded; and in other specific implementations of the embodiments of the present disclosure, the first re-recording task and the second re-recording task can be executed at the same time as needed, that is, the simultaneous re-recording of the two scenes of transmission damage and video missing can be realized, which is not limited here. Figure 1The supplementary recording task in step 200 and step 300 may be the first supplementary recording task, or the first supplementary recording task and the second supplementary recording task may be executed simultaneously.

[0076] It is understandable that, through the configuration of the second supplementary recording task, it is possible to effectively deal with the back-end media stream loss scenario caused by factors such as network disconnection and back-end storage failure, thereby realizing the automatic detection and supplementary recording of media stream transmission in common application scenarios such as network congestion packet loss, video transmission loss, and video transmission damage that affect the integrity of the video. It is understandable that the judgment and supplementary recording action of the above-mentioned media stream loss and the judgment and supplementary recording action of the damaged media stream transmission provided in the above-mentioned embodiment can be selected or executed simultaneously according to actual needs, and no limitation is made here.

[0077] In the above embodiment, specifically, each first supplementary recording task includes configured supplementary recording plan parameters, and the corresponding supplementary recording task is generated by scanning and detecting the video record through the supplementary recording plan parameters. The supplementary recording plan parameters may include the front-end device information corresponding to the media stream that needs to be supplemented, the supplementary recording start and end time information of the supplementary recording media stream, the supplementary recording execution time, and the supplementary recording execution priority. It can be understood that the supplementary recording execution time may be executed immediately after the integrity detection is missing and / or damaged, or it may be executed regularly according to the preset time. Considering that the execution of the supplementary recording task may increase the network transmission burden between the front-end device and the back-end server, in order to ensure the supplementary recording quality and the supplementary recording success rate of the supplementary recording task, and at the same time avoid obstruction and damage to other transmission media streams, in an optional specific implementation of the present disclosure, the supplementary recording execution time can be configured to be executed regularly during the preset off-peak period. If a single occasional supplementary recording task does not formulate the supplementary recording execution time in the supplementary recording plan parameters, it can be controlled by the global configuration parameters preset by the back-end server.

[0078] In the above embodiment, further, in the specific implementation of the aforementioned step 300, a corresponding supplementary recording execution plan can be generated according to all the first supplementary recording tasks to be executed corresponding to a certain front-end device, wherein each supplementary recording task in the supplementary recording execution plan can be controlled according to the priority, wherein the priority is related to dimensions such as the duration and degree of damage of the media stream to be supplemented, wherein the measurement of the degree of damage preferably adopts the damaged video duration indicator of the media stream, that is, the priority of the supplementary recording task is positively correlated with the damaged video duration of the media stream.

[0079] In the traditional video re-recording solution, after a network transmission abnormality occurs, a retransmission-type re-recording is immediately initiated by immediately requesting a segment of frame data in the cache of the front-end device for re-recording. This solution is not flexible enough and the execution timing of the re-recording task cannot be set. It is easy to increase the network burden when the network is congested, forming a vicious circle.

[0080] In the above embodiment, further, in the aforementioned step 400, during the execution of the first supplementary recording task, it is also necessary to obtain and continuously monitor the network transmission environment between the front-end device and the back-end server: when the network transmission environment does not support the supplementary recording execution plan, that is, when the network is severely congested, there are too many supplementary recording tasks to be completed, and it is expected that they cannot be completed, the supplementary recording tasks are executed in sequence according to the supplementary recording execution priority, that is, the supplementary recording tasks with more serious damage are executed first according to the priority level to ensure that the media stream obtained by the back-end server is relatively complete in the large-scale time dimension. In this embodiment, supplementary recording can be executed in sequence according to priority and the supplementary recording execution plan is supported, so that the execution timing of the supplementary recording task is accurately controlled during the video supplementary recording process, and network congestion will not occur.

[0081] In the above embodiment, further, in the case of a poor network transmission environment, the number of concurrent executions of the first supplementary recording task can be adjusted to avoid network overload, which affects the media transmission quality of the video recording task and the supplementary recording task. Specifically, the control of the number of concurrent executions of the first supplementary recording task can be divided according to the area where the front-end device is located, so as to facilitate more precise control to cope with different network bandwidths and network congestion scenarios from different front-end device areas to the back-end central computer room. It can be understood that the above-mentioned priority setting and the specific execution configuration of the first supplementary recording task can also be applied to the configuration and execution of the second supplementary recording task, and no limitation is made here.

[0082] In the above embodiment, in the specific implementation of the aforementioned step 400, in order to ensure that the supplementary video clip is accurately aligned with the original video clip, or seamlessly connected with the previous and next clips of the original video, it is necessary to maintain the clock synchronization between the front-end device and the back-end server, and at the same time, the time range of the specified historical media stream is as accurate as possible to the millisecond level; in addition, in order to deal with the problem that some front-end devices do not accurately reach the deadline when sending the supplementary media stream and stop sending the code stream in advance, a margin of no more than the duration of one image group can be added to the deadline of the requested historical media stream as the excess deadline, and this margin can usually be configured to be 1 to 3 seconds. When receiving the supplementary media stream, the duration of the received supplementary media stream is calculated by using information such as the timestamp interval. When the duration reaches the requested excess deadline, the last key frame in the historical media stream is used as the supplementary cutoff position, and the historical media stream is cropped according to the supplementary cutoff position to obtain the supplementary media stream, and the key frame is not written into the segment corresponding to the supplementary media stream. It is understandable that the use of key frames as matching points for the supplementary recording end point is based on the premise that the segmented media streams corresponding to each segmented data block use key frames as starting frames. During the transmission of the historical media stream, the transmission method can be configured to use a reliable transmission protocol based on TCP, thereby further improving the execution reliability of the supplementary recording task.

[0083] In some embodiments of the present disclosure, based on the completion of the supplementary recording task, it is necessary to further perform supplementary recording verification on the execution result of the supplementary recording task to ensure the accuracy and reliability of the supplementary recording result. In an optional specific implementation of the present disclosure, for the supplementary recording media stream obtained by the supplementary recording task, it can be encapsulated and stored in segments in the same way as the original media stream, and the obtained supplementary recording data block is compared with the segmented data block corresponding to the original media stream: if the size of the encapsulated data block corresponding to the video segment obtained by the supplementary recording is equal to the size of the encapsulated data block corresponding to the original video segment, it is determined that the corresponding supplementary recording task has passed the supplementary recording verification; if the size of the encapsulated data block corresponding to the video segment obtained by the supplementary recording is smaller than the size of the encapsulated data block corresponding to the original video segment, it is determined that the corresponding supplementary recording task has not passed the supplementary recording verification, and the corresponding supplementary recording media stream will not be adopted. At this time, the corresponding supplementary recording task can be reconfigured and executed according to the previous integrity detection result. Those skilled in the art can also select other suitable supplementary recording verification methods according to actual needs, such as repeating the relevant steps of the aforementioned integrity detection, etc., which are not limited here.

[0084] In the above embodiment, further, when the supplementary recording task is completed and the supplementary recording verification is passed, the original segmented data blocks and their corresponding video records that are damaged and / or missing can be deleted and replaced with the supplementary recording data blocks to achieve complete video storage of the media stream. In another optional specific implementation, the supplementary recording data blocks and the original segmented data blocks can be stored at the same time, and additional retrieval marks can be added to the original segmented data blocks that are damaged and / or missing. When the user searches for video segments through the video records, the video segments corresponding to the supplementary recording data blocks are displayed by default. Only through specific additional retrieval marks can the original segmented data blocks be obtained. The above scheme can be applied to special application scenarios with high data security requirements, and the original video information that is damaged and / or missing can be retained for users to back up and view when needed.

[0085] In some embodiments of the present disclosure, a video surveillance system is also provided, which includes multiple front-end devices and a back-end server. The back-end server is configured with multiple storage spaces for responding to recording requests, obtaining and storing media streams transmitted by each front-end device. The back-end server is configured with a functional module capable of executing the supplementary recording method provided in the aforementioned embodiment, which is used to achieve complete acquisition of the media streams transmitted by each front-end device, which will not be described in detail here.

[0086] The embodiment of the present disclosure also provides a computer-readable storage medium for storing a program, and the steps of the video data supplementary recording method are implemented when the program is executed. In some possible implementations, various aspects of the present disclosure can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary implementations of the present disclosure described in the above-mentioned video data supplementary recording method section of this specification.

[0087] Specifically, Figure 2 According to an embodiment of the present disclosure, a schematic diagram of the structure of a computer-readable storage medium is shown. Figure 2 As shown, a program product 400 for implementing the above-mentioned video data supplementary recording method according to an embodiment of the present disclosure is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0088] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0089] Computer readable storage media may include data signals propagated in baseband or as part of a carrier wave, in which readable program codes are carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0090] The program code for executing the specific implementation operations of the video data supplementation method provided by the aforementioned embodiments of the present disclosure can be written in any combination of one or more programming languages, and the programming languages ​​include object-oriented programming languages—such as Java, C++, etc., and also include conventional procedural programming languages—such as "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0091] In summary, the technical solution provided by the present disclosure can detect the integrity of the video in real time from the two aspects of damage and missing during the synchronous storage of the media stream at the back end, and can more accurately and intuitively reflect the integrity of the media stream storage. The damage detection of the media stream includes means such as packet loss detection and anti-tampering detection of the media stream. At the same time, multi-dimensional evaluation indicators such as the number of packet losses, packet loss rate, and damage duration are introduced to be closer to the user's experience and perception. It can realize automatic detection and re-recording of media stream transmission for common application scenarios such as network congestion packet loss, video transmission damage, network interruption, storage anomalies, etc. that affect the integrity of the video, and can adjust the setting and execution of the re-recording task according to changes in the network transmission environment, which has promotional value.

[0092] The above content is a further detailed description of the present disclosure in combination with specific preferred implementation modes, and it cannot be determined that the specific implementation of the present disclosure is limited to these descriptions. For ordinary technicians in the technical field to which the present disclosure belongs, several simple deductions or substitutions can be made without departing from the concept of the present disclosure, which should be regarded as falling within the scope of protection of the present disclosure.

Claims

1. A video data supplementary recording method, applied to a back-end server in a video surveillance system, wherein the back-end server responds to a video recording request, obtains a media stream transmitted by a front-end device and stores the media stream; It is characterized in that The video data supplementary recording method comprises the following steps: Performing integrity detection on the acquired media stream to obtain a transmission quality status of the media stream, wherein the transmission quality status includes a transmission damage status of the media stream; Generate a corresponding first supplementary recording task according to the transmission damage situation and a preset video data supplementary recording plan, where the first supplementary recording task covers a corresponding time period in the media stream where the transmission damage situation exists; The first supplementary recording task is executed to enable the back-end server to completely acquire the media stream.

2. The video data supplementary recording method according to claim 1, characterized in that: The process of storing the acquired media stream includes the following steps: Encapsulating the acquired media stream and segmenting it according to a fixed duration to form a plurality of segmented data blocks and storing them in the back-end server, and generating a video record corresponding to each segmented data block; The video record includes at least one or a combination of multiple items of the front-end device information, segment time information, storage location information and the transmission quality status corresponding to each segment data block.

3. The video data supplementary recording method according to claim 2, characterized in that: In the process of generating the corresponding first supplementary recording task according to the transmission damage situation and the preset video data supplementary recording plan, the following steps are included: Scan all the video records corresponding to the media stream to be detected to obtain the damaged video duration corresponding to each of the video records, wherein the damaged video duration includes the sum of the damaged durations of each image group in the segmented media stream corresponding to the video record; Based on the video data re-recording plan, when the duration of the damaged video is greater than a first preset threshold, the first re-recording task is generated, and the first re-recording task includes re-acquiring from the front-end device all historical media streams corresponding to the segmented data blocks containing the damaged frames.

4. The video data supplementary recording method according to claim 3, characterized in that: The process of obtaining the damaged duration of each image group in the segmented media stream includes the following steps: Based on the transmission quality status corresponding to the segmented media stream in the video record, obtaining the distribution of damaged frames in each image group; The duration from the first damaged frame to the next key frame in each of the image groups is used as the damaged duration of the image group.

5. The video data supplementary recording method according to any one of claims 1 to 4, characterized in that: The integrity detection includes performing packet loss detection on the media stream to be detected and / or performing tamper-proof verification on built-in tamper-proof information of the media stream to be detected; Obtaining a distribution of damaged frames of the to-be-detected media stream based on the packet loss detection and / or the anti-tampering verification; Obtaining the number of packet losses and / or the packet loss rate of the media stream to be detected based on the packet loss detection; In the process of generating the first supplementary recording task, based on the video data supplementary recording plan, when the number of packet losses is greater than a second preset threshold and / or the packet loss rate is greater than a third preset threshold, the corresponding first supplementary recording task is generated.

6. The video data supplementary recording method according to claim 2, characterized in that: The video data supplementary recording method further includes: Scan all the video records corresponding to the media stream to be detected to obtain the start timestamps and end timestamps corresponding to all the video records; According to the start timestamp and the end timestamp, determine whether the media stream to be detected is continuous: if not, obtain the missing time period corresponding to the media stream to be detected; Based on the video data re-recording plan, when the duration of the missing period is greater than a fourth preset threshold, a corresponding second re-recording task is generated, and the second re-recording task includes re-acquiring the historical media streams corresponding to all the video records associated with the missing period from the front-end device.

7. The video data supplementary recording method according to claim 1, characterized in that: Each of the first supplementary recording tasks includes front-end device information of the supplementary recording media stream, supplementary recording start and end time information of the supplementary recording media stream, and supplementary recording execution priority; The priority of executing the supplementary recording is positively correlated with the duration of the damaged video of the media stream; The process of performing the first supplementary recording task includes: Acquire the first supplementary recording task to be executed to generate a corresponding supplementary recording execution plan; Acquire the network transmission environment between the front-end device and the back-end server; If the network transmission environment does not support the supplementary recording execution plan, executing the first supplementary recording tasks in sequence according to the supplementary recording execution priority; and / or The number of the first supplementary recording tasks to be executed concurrently is adjusted.

8. The video data supplementary recording method according to claim 7, characterized in that: The segmented media stream corresponding to each segmented data block uses a key frame as a starting frame; The performing of the first supplementary recording task comprises the following steps: Maintaining clock synchronization between the front-end device and the back-end server; According to the supplementary recording start and end time information of the supplementary recording media stream, a preset margin is added to obtain an excess deadline corresponding to the supplementary recording media stream, wherein the preset margin is less than the duration of a single image group in the supplementary recording media stream; Based on the supplementary recording start and end time information and the overrun cut-off time, reacquire the historical media stream from the front-end device, and use the last key frame in the historical media stream as the supplementary recording cut-off position; The historical media stream is trimmed according to the supplementary recording cutoff position to obtain the supplementary recording media stream.

9. A video surveillance system, characterized in that: include: Front-end equipment; The back-end server is used to respond to the recording request, obtain the media stream transmitted by the front-end device and store it; The backend server is also used to adopt the video data supplementary recording method described in any one of claims 1 to 8 to achieve complete acquisition of the media stream.

10. A computer-readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps of the video data supplementary recording method described in any one of claims 1 to 8 are implemented.

Images