Video storage method and device and storage medium

Abstract

The invention discloses a video storage method and device and a storage medium, and the method comprises the steps: predicting the position of an initial key frame of a next picture group, and obtaining a predicted position; calculating to obtain a predicted cache space based on the cache space required by the image frame between the current image frame position and the predicted position; if the size relationship between the current residual space of the buffer and the predicted buffer space meets a preset condition, responding to the completion of encoding of an initial key frame of a next picture group, and starting to write data of a new video file into the buffer by taking the key frame as a starting point, and taking the cache data before the cache as the data of the old video file and transferring the data to the persistent memory. By predicting the position of the initial key frame of the next picture group and sensing the opportunity of video slicing in advance, video slicing is accurately carried out at the position of the key frame, each obtained video file can be played independently, redundant key frames do not exist, and the storage efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to a video storage method, device and storage medium. Background Technology

[0002] After the camera captures image data, it encodes the data and temporarily stores the encoded image data (including keyframes and non-keyframes) in the camera's internal buffer. When the buffer is full or certain conditions are met, the buffered data is written to external storage to form an independent video file. This process is called video segmentation.

[0003] If the starting frame of the video file obtained by video segmentation is not a keyframe (such as an I-frame), but a non-keyframe (such as a P-frame) that requires reference to other image frame data, the player will experience a black screen, stuttering, or decoding errors when opening the video file. It will not be able to decode and display normally until the first keyframe in the video file is encountered, which seriously affects the user experience and the validity of video evidence.

[0004] To address the aforementioned issues, existing solutions typically involve inserting I-frames at fixed intervals or forcibly encoding I-frames during video segmentation. This abruptly disrupts the original Group of Pictures (GOP) structure, inserts redundant I-frame data, increases the overall video bitrate, and wastes storage space. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides at least one video storage method, device, and storage medium.

[0006] The first aspect of this application provides a video storage method, which includes: encoding real-time acquired image data and storing it in a buffer, and predicting the position of the keyframe at the beginning of the next image group to obtain the predicted position; calculating the predicted buffer space based on the buffer space required for image frames between the current image frame position and the predicted position, and obtaining the current remaining space of the buffer; if the size relationship between the current remaining space and the predicted buffer space meets a preset condition, then in response to the completion of the encoding of the keyframe at the beginning of the next image group, writing data of a new video file to the buffer starting from the keyframe, and transferring the buffered data from before to the buffer as data of the old video file to persistent storage.

[0007] In one embodiment, predicting the position of the keyframe at the beginning of the next image group to obtain the predicted position includes: statistically analyzing the positions of keyframes in the cached image group to obtain keyframe position patterns; and / or, performing abrupt change prediction based on the image content of real-time acquired image data to obtain abrupt change prediction results; and predicting the position of the keyframe at the beginning of the next image group based on the keyframe position patterns and / or abrupt change prediction results to obtain the predicted position.

[0008] In one embodiment, the prediction cache space is calculated based on the cache space required for image frames between the current image frame position and the prediction position. This includes: calculating the number of image frames between the current image frame position and the prediction position, and obtaining a preset safety margin frame number; summing the number of image frames and the safety margin frame number to obtain the prediction cache space.

[0009] In one embodiment, the step of generating the keyframe for the start of the next image group includes: calculating the number of image frames between the current image frame position and the predicted position; calculating the duration of the old video file based on the number of image frames and the frame interval time, and obtaining a preset maximum duration; comparing the duration of the old video file with the maximum duration to obtain a time comparison result; determining the matching video segmentation pattern based on the time comparison result, and executing the decision corresponding to the matching video segmentation pattern to generate the keyframe for the start of the next image group.

[0010] In one embodiment, determining a matching video segmentation pattern based on the time comparison result and executing the decision corresponding to the matching video segmentation pattern to generate the keyframe for the start of the next image group includes: if the duration of the old video file is less than or equal to the maximum duration, then a perfect segmentation pattern is adopted. The decision for the perfect segmentation pattern includes: waiting for the image data corresponding to the predicted position to be encoded as a keyframe to obtain the keyframe for the start of the next image group; if the duration of the old video file is greater than the maximum duration, then a fast segmentation pattern is adopted. The decision for the fast segmentation pattern includes: immediately using the current cached data of the buffer as the old video file and using the keyframe obtained by the next normal encoding as the keyframe for the start of the next image group.

[0011] In one embodiment, the step of generating the keyframe for the start of the next image group includes: comparing the current remaining space with a preset remaining space threshold to obtain a spatial comparison result; determining a matching video segmentation pattern based on the spatial comparison result; and executing the decision corresponding to the matching video segmentation pattern to generate the keyframe for the start of the next image group.

[0012] In one embodiment, the video slicing mode includes a perfect slicing mode and a fast slicing mode. The remaining space thresholds are sorted from largest to smallest, including a first threshold and a second threshold. Determining the matching video slicing mode based on the space comparison result includes: if the current remaining space is greater than the first threshold, comparing the duration of the old video file and the maximum duration; determining whether to use the perfect slicing mode or the fast slicing mode based on the time comparison result; if the current remaining space is less than the first threshold but greater than the second threshold, increasing the prediction buffer space and / or decreasing the maximum duration, then comparing the relationship between the current remaining space and the prediction buffer space, and comparing the relationship between the duration of the old video file and the maximum duration, determining whether to use the perfect slicing mode or the fast slicing mode based on the comparison result; if the current remaining space is less than the second threshold, reducing the encoding resolution and / or frame rate of the image data, and using the fast slicing mode.

[0013] In one embodiment, the method further includes: dividing the buffer into: an old block area, a transition area, and a new block preparation area; the old block area is used to store image frames that will be written to the old video file; the transition area is used to store subsequent image frames of the current image group, and the size of the transition area is related to the prediction buffer space; the new block preparation area is the reserved space required for the starting keyframe and subsequent image frames of the new video file.

[0014] In one embodiment, the method further includes: performing scene change analysis on the real-time acquired image data to classify the current scene type; if the current scene type is a static scene, then extending the image group length; if the current scene type is a dynamic scene, then shortening the image group length.

[0015] A second aspect of this application provides a video storage device, comprising: a caching and prediction module, used to encode real-time acquired image data and store it in a cache, and predict the position of the keyframe at the beginning of the next image group to obtain the predicted position; a cache space determination module, used to calculate the predicted cache space based on the cache space required for image frames between the current image frame position and the predicted position, and obtain the current remaining space of the cache; and a video segmentation module, used to, in response to the completion of the encoding of the keyframe at the beginning of the next image group, write data of a new video file to the cache starting from the keyframe, and transfer the cached data in the cache before the keyframe to persistent storage as data of the old video file, if the size relationship between the current remaining space and the predicted cache space meets a preset condition.

[0016] A third aspect of this application provides an electronic device, including a memory and a processor, wherein the processor is configured to execute program instructions stored in the memory to implement the video storage method described above.

[0017] The fourth aspect of this application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement the video storage method described above.

[0018] The above scheme encodes the real-time acquired image data and stores it in a buffer. It then predicts the position of the keyframe at the beginning of the next image group to obtain the predicted position. Based on the buffer space required for the image frames between the current image frame position and the predicted position, it calculates the predicted buffer space and obtains the current remaining space in the buffer. If the size relationship between the current remaining space and the predicted buffer space meets a preset condition, in response to the completion of the keyframe encoding at the beginning of the next image group, it starts writing data for the new video file to the buffer starting from the keyframe. The buffer's previous cached data is transferred to persistent storage as data for the old video file. By ensuring that each new video file starts with a keyframe, the scheme predicts the position of the keyframe at the beginning of the next image group to detect the timing of video segmentation in advance, allowing for precise video segmentation at the keyframe position. Each resulting video file can be played independently without redundant I-frames, thus improving storage efficiency.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0021] Figure 1 This is a schematic diagram illustrating the implementation environment of the solution in an exemplary embodiment of this application; Figure 2 This is a flowchart illustrating a video storage method in an exemplary embodiment of this application; Figure 3 This is a schematic diagram illustrating the partitioning of a cache, as shown in an exemplary embodiment of this application; Figure 4 This is a flowchart illustrating a video storage method in another exemplary embodiment of this application; Figure 5 This is a schematic diagram of a video storage system illustrating an exemplary embodiment of this application; Figure 6 This is a block diagram illustrating a video storage device according to an exemplary embodiment of this application; Figure 7 This is a schematic diagram of the structure of an electronic device shown in an exemplary embodiment of this application; Figure 8This is a schematic diagram illustrating the structure of a computer-readable storage medium, as shown in an exemplary embodiment of this application. Detailed Implementation

[0022] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0023] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0024] In this document, the term "and / or" is merely a description of the association information of related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0025] The video storage method provided in the embodiments of this application will be described below.

[0026] Please refer to Figure 1 , Figure 1 This is a schematic diagram illustrating an implementation environment of the scheme according to an exemplary embodiment of this application. The implementation environment may include a video acquisition terminal, which is used to acquire image data and encode it into a video stream.

[0027] The video acquisition device can be any device used for image acquisition, such as a PTZ camera, panoramic camera, or dome camera; this application does not limit this.

[0028] The video capture terminal is equipped with a buffer, which is usually located in memory. It is a high-speed temporary storage device with fast data read and write speed, but it is a volatile storage medium and the data will be lost after power failure or restart.

[0029] The video capture terminal is also connected to persistent storage, which is usually based on disk or solid-state drive. Compared with the buffer, persistent storage has a slower data read and write speed, but persistent storage is a non-volatile storage medium that ensures that data is not lost after power failure.

[0030] It should be noted that persistent storage can be integrated into the video capture terminal or set up independently of the video capture terminal, such as when persistent storage is deployed in a cloud server. This application does not limit this.

[0031] The video storage method provided in this application can be executed by a video acquisition terminal, or by other electronic devices that are connected to the video acquisition terminal, such as a server, or by the video acquisition terminal and other electronic devices interacting and cooperating. This application does not limit the scope of execution.

[0032] In one embodiment, the video capture device is a camera that supports Always On Video (AOV). In AOV mode, the camera captures image data at low frame rate intervals (e.g., 1-5 fps) for encoding and temporarily stores the encoded image frames in an internal buffer. The video segmentation process includes: when the buffer reaches a certain condition, the system wakes up the main control unit and writes the data in the buffer to an external memory (e.g., an SD (Secure Digital) card) to form an independent video file.

[0033] Please see Figure 2 , Figure 2 This is a flowchart illustrating a video storage method in an exemplary embodiment of this application, as shown below. Figure 2 As shown, the video storage method includes at least steps S210 to S230, which are described in detail below: Step S210: Encode the real-time acquired image data and store it in the buffer, and predict the position of the keyframe at the beginning of the next image group to obtain the predicted position.

[0034] The video capture device acquires image data in real time, and then encodes the acquired image data using a preset video encoding method.

[0035] Video encoding refers to the method of converting a file in an original video format into another video format file through compression technology. The corresponding encoding and decoding standards include H.261, H.263, and H.264 of the International Telecommunication Union. This application does not limit the video encoding method used.

[0036] By encoding image data, keyframes or non-keyframes can be obtained. Keyframes are image frames that can be encoded and decoded independently, while non-keyframes are image frames that need to be encoded and decoded by referring to other image frames.

[0037] For example, keyframes include intra-coded pictures (I-frames), which contain complete image information and can be encoded and decoded independently without relying on other frames; non-keyframes include predicted frames (P-frames) and / or bi-predicted frames (B-frames). P-frames are frames that need to be encoded and decoded based on the previous I-frame or P-frame, while B-frames are frames that need to be encoded and decoded by referring to both the previous and next I-frames or P-frames.

[0038] A Group of Pictures (GOP) is a series of consecutive frames in video encoding, consisting of a starting keyframe and subsequent non-keyframes.

[0039] The encoded image frames are stored in a buffer. In addition, the position of the keyframe at the beginning of the next image group is predicted, and the predicted position reflects how many other image frames need to be encoded to encode the next keyframe.

[0040] The predicted position can be obtained by predicting the keyframe position of the next group of images based on the pattern of the group of images (GOP) and / or the prediction results of future image changes.

[0041] Step S220: Calculate the prediction cache space based on the cache space required for the image frames between the current image frame position and the prediction position, and obtain the current remaining space of the cache.

[0042] The current image frame position refers to the position of the image frame currently being encoded. Based on the current image frame position and the prediction position, we can determine how many other image frames need to be encoded to encode the next keyframe. Based on the number of other image frames that still need to be encoded, we can calculate the prediction buffer space.

[0043] Predicted cache space refers to the predicted cache space size required to store the current image group, or the predicted cache space size required to store the current image group and the next image group.

[0044] Additionally, the usage of the cache is read to obtain the current remaining space of the cache.

[0045] It should be noted that the predicted cache space and the current remaining space can be represented by the number of image frames or by the specific data size; this application does not limit this.

[0046] Step S230: If the size relationship between the current remaining space and the predicted cache space meets the preset conditions, then in response to the completion of the keyframe encoding at the beginning of the next image group, the data of the new video file is written to the cache starting from the keyframe, and the cached data in the cache before is transferred to the persistent storage as the data of the old video file.

[0047] Comparing the current remaining space with the predicted cache space, their size relationship can reflect the urgency of video segmentation.

[0048] If the size relationship between the current remaining space and the predicted cache space meets the preset conditions, then video segmentation will begin.

[0049] It can detect whether the current remaining space is less than or equal to the predicted cache space, or detect whether the difference between the current remaining space and the predicted cache space is less than a preset difference threshold, to determine whether their size relationship meets the preset conditions.

[0050] For example, the predicted cache space refers to the predicted cache space size required to store the current image group and the next image group. If the current remaining space is less than or equal to the predicted cache space, it is determined that the size relationship meets the preset condition.

[0051] For example, the predicted cache space refers to the predicted cache space size required to store the current image group. If the difference between the current remaining space and the predicted cache space is less than or equal to a preset difference threshold, it is determined that the size relationship meets the preset condition.

[0052] If video segmentation is required, in response to the completion of keyframe encoding at the beginning of the next image group, data for the new video file is written to the buffer starting from the keyframe, and the previously cached data in the buffer is transferred to persistent storage as data for the old video file, thus completing the video segmentation.

[0053] Current technologies typically trigger video segmentation when the cache is full. If the current image group is not yet finished, I-frames need to be forcibly encoded, and redundant I-frame data will increase the overall video bitrate. Alternatively, instead of forcibly encoding I-frames, a dependency relationship is established between video files. When a new video file is played after an old video file has finished playing, since the first frame of the new video file is not a keyframe, the last keyframe of the old video file is looked up based on the dependency relationship and used as the keyframe of the new video file. This destroys the independence of video files. If the old video file is lost or damaged, the new video file will not be able to play, resulting in low reliability.

[0054] The video storage method provided in this application encodes real-time acquired image data and stores it in a buffer. It predicts the position of the keyframe at the beginning of the next image group to obtain the predicted position. Based on the buffer space required for the image frames between the current image frame position and the predicted position, it calculates the predicted buffer space and obtains the current remaining space of the buffer. If the size relationship between the current remaining space and the predicted buffer space meets a preset condition, in response to the completion of the encoding of the keyframe at the beginning of the next image group, it starts writing data of the new video file to the buffer from the keyframe and transfers the previous buffer data to the persistent storage as the data of the old video file. Under the premise of ensuring that each new video file starts with a keyframe, by predicting the position of the keyframe at the beginning of the next image group, it can perceive the timing of video segmentation in advance, so as to accurately segment the video at the keyframe position. Each video file can be played independently and there are no redundant I-frames, thus improving storage efficiency.

[0055] The following describes some embodiments of this application in detail.

[0056] In one implementation, the length of the image group can also be adjusted.

[0057] For example, scene change analysis can be performed on real-time acquired image data to classify the current scene type; if the current scene type is a static scene, the length of the image group can be increased; if the current scene type is a dynamic scene, the length of the image group can be shortened.

[0058] Scene change analysis is performed on the real-time acquired image data to obtain scene analysis results.

[0059] Scene change analysis can be achieved using methods such as frame difference analysis, optical flow analysis, or machine learning; this application does not limit the specific methods used.

[0060] Scene analysis results reflect the degree of change in image content between consecutive image frames. If the degree of change in image content is high, the current scene type is a dynamic scene; if the degree of change in image content is low, the current scene type is a static scene.

[0061] If the current scene type is a static scene, the GOP length is increased; if the current scene type is a dynamic scene, the GOP length is shortened. By dynamically adjusting the GOP length, bitrate can be saved and block segmentation flexibility can be improved.

[0062] For example, in static scenes, extending the GOP length from 30 frames to 60 frames reduces the number of I-frames, thereby significantly improving the video compression rate and saving storage space while maintaining basic image quality. In dynamic scenes, shortening the GOP length from 30 frames to 15 frames not only improves the encoding quality of dynamic images but also creates more potential I-frame segmentation points for subsequent video segmentation, increasing system flexibility.

[0063] In one implementation, to facilitate cache space management, the cache is logically dynamically partitioned rather than physically fixed.

[0064] Specifically, the buffer is divided into: an old block area, a transition area, and a new block preparation area; the old block area is used to store image frames that will be written to the old video file; the transition area is used to store subsequent image frames of the current image group, and the size of the transition area is related to the prediction buffer space; the new block preparation area is the reserved space required for the starting keyframe and subsequent image frames of the new video file.

[0065] For example, please see Figure 3 , Figure 3 This is a schematic diagram illustrating the partitioning of a cache, as shown in an exemplary embodiment of this application. Figure 3As shown, a cache mapping table is maintained in memory, which logically divides the cache into: old block area, transition area and new block preparation area. The above partitions are flexibly and in real time based on predicted location, predicted cache space and other factors.

[0066] Specifically, the old block area is used to store image frames that will be written to the old video file. The transition area is used to store subsequent image frames of the current image group (which belongs to the old video file); it is a flexible buffer whose size is related to the prediction buffer space. The new block preparation area is reserved space to ensure the generation of the starting keyframe and subsequent image frames of the new video file.

[0067] The above partitioning method enables fine-grained management of cached resources in the cache, facilitating video segmentation.

[0068] When performing video segmentation, the position of the keyframe at the beginning of the next image group is predicted to obtain the predicted position.

[0069] In one embodiment, it can be determined whether the current remaining space of the buffer is less than a preset space threshold. If it is less, then video segmentation judgment is performed, and the steps of predicting the position of the keyframe at the beginning of the next image group and subsequent steps are executed.

[0070] In one embodiment, step S210 predicts the position of the keyframe at the beginning of the next image group to obtain the predicted position, including the following steps S211 to S212.

[0071] Step S211: Statistically determine the positions of keyframes in the cached image group to obtain the keyframe position patterns; and / or, perform mutation prediction based on the image content of the real-time acquired image data to obtain mutation prediction results.

[0072] Step S212: Based on the keyframe position pattern and / or mutation prediction results, predict the position of the starting keyframe of the next image group to obtain the predicted position.

[0073] For example, by observing the positions of the I-frames in the first few GOPs, we can infer the pattern of keyframe positions and thus predict the position of the keyframe at the beginning of the next group of images.

[0074] For example, by obtaining the length of the preceding GOP and calculating its average length, we can obtain the current GOP length. Given the current I-frame position (frame t) and the current GOP length (N), we can directly predict that the next I-frame will appear in frame t+N.

[0075] For example, the content of an image frame can be analyzed to determine whether an I-frame should appear.

[0076] For example, calculate the differences between consecutive frames, such as mean absolute difference (MAD) and mean squared error (MSE), or extract the color histogram, edge features, and depth features of image frames. Based on the extracted features, predict the moment when the image changes drastically, obtain the mutation prediction result, and predict the position of the keyframe at the beginning of the next image group based on the mutation prediction result.

[0077] For example, by combining the keyframe position patterns and mutation prediction results, the position of the next I-frame can be predicted. This can be done by calculating the average of the two prediction results or selecting the maximum or minimum value of the two prediction results to obtain the position of the keyframe at the beginning of the next image group, and outputting a confidence score to evaluate the prediction reliability.

[0078] Then, the predicted cache space is calculated based on the predicted location.

[0079] In one embodiment, step S220 calculates the prediction cache space based on the cache space required for image frames between the current image frame position and the prediction position, including the following steps S221 to S222.

[0080] Step S221: Calculate the number of image frames between the current image frame position and the predicted position, and obtain the preset safety margin frame number.

[0081] Step S222: Sum the number of image frames and the number of safety margin frames to obtain the prediction buffer space.

[0082] The number of safety margin frames can be preset based on experience or calculated flexibly.

[0083] For example, the number of safety margin frames can be flexibly set based on the confidence level corresponding to the predicted location, and / or the accuracy of historical video segments, the size of the current remaining space, etc.

[0084] The prediction buffer space is obtained by summing the number of image frames between the current image frame position and the prediction position and the number of safety margin frames. This prediction buffer space reflects how many image frames still need to be stored after storing the current GOP and the next GOP.

[0085] In the above embodiments, the prediction cache space and the current remaining space are represented by the number of image frames. Of course, the actual data size can also be used to represent the prediction cache space and the current remaining space.

[0086] For example, the size of the most recent preset number of image frames can be obtained by sliding a window. Based on the size of these image frames, the average image frame size can be calculated. Then, based on the average image frame size and the number of image frames between the current image frame position and the prediction position, the prediction buffer space can be calculated.

[0087] For example, the most recent 50 frames are obtained based on the sliding window. The average P-frame size is calculated in real time based on the size of these 50 frames to obtain Size_p. The average I-frame size is calculated to obtain Size_i. The number of safety margin frames is obtained. The number of image frames and the number of safety margin frames are summed and multiplied with Size_p. The result of the multiplication is summed with Size_i to obtain the prediction cache space. This prediction cache space reflects how much storage space is needed to store the current GOP and the next GOP.

[0088] Subsequently, by comparing the size relationship between the current remaining space and the predicted cache space, it is determined whether to perform video segmentation.

[0089] For example, if the number of image frames between the current image frame position and the predicted position is n_pre, the number of safety margin frames is n_thr, and the current remaining space is n_rem, and n_rem≤(n_pre+n_thr), then video segmentation preparation is triggered. At this point, segmentation does not begin immediately; instead, subsequent P-frames of the current GOP are written to the "old block area" until the next predicted I-frame is encoded and stored in the buffer. All data from the start point to the last P-frame in the buffer is then written to the old video file. Finally, using the previously buffered I-frame as the starting point, the writing of the new video file begins.

[0090] In one implementation, the specific method of video segmentation can be determined according to the actual situation, that is, the method of generating the keyframe for the beginning of the next image group can be determined.

[0091] For example, based on the duration of the old video file and / or the current remaining space, determine how to generate the keyframe for the start of the next image group, so as to generate the keyframe for the start of the next image group.

[0092] Example 1: The steps for generating the keyframe for the start of the next image group include the following steps S2311 to S2314.

[0093] Step S2311: Calculate the number of image frames between the current image frame position and the predicted position.

[0094] Step S2312: Based on the number of image frames and the frame interval time, calculate the duration of the old video file and obtain the preset maximum duration.

[0095] Specifically, the frame interval time is obtained by multiplying the number of image frames and the frame interval time to calculate the duration of the old video file.

[0096] For example, if it is calculated that N_p P frames still need to be encoded before the next I frame is encoded, and the frame interval is t seconds, then the continuation time of the old video file is T_continue = N_p * t.

[0097] The maximum duration T_max can be preset based on experience, or it can be flexibly set based on the confidence level of the predicted position, and / or the accuracy of historical video segments, the size of the current remaining space, etc.

[0098] Step S2313: Compare the duration of the old video file with the maximum duration to obtain the time comparison result.

[0099] Step S2314: Determine the matching video segmentation pattern based on the time comparison result, and execute the decision corresponding to the matching video segmentation pattern to generate the keyframe for the start of the next image group.

[0100] For example, if the duration of the old video file is less than or equal to the maximum duration, the perfect slicing mode is used. The decision of the perfect slicing mode includes: waiting for the image data corresponding to the predicted position to be encoded as a keyframe to obtain the keyframe for the start of the next image group; if the duration of the old video file is greater than the maximum duration, the fast slicing mode is used. The decision of the fast slicing mode includes: immediately using the current cached data of the buffer as the old video file, and using the keyframe obtained by the next normal encoding as the keyframe for the start of the next image group.

[0101] Specifically, two video segmentation modes are set: Perfect Slicing Mode: Wait for the image data corresponding to the predicted position to be encoded as a keyframe to obtain the keyframe for the start of the next image group.

[0102] The perfect chunking mode is triggered when: T_continue ≤ T_max; If the above triggering conditions are met, it indicates that the duration of the old video file caused by waiting for the next I-frame is within an acceptable range, and pursuing storage efficiency and GOP integrity is worthwhile. At this time, continue to write the subsequent P-frames of the current GOP into the "old block area" until the next predicted I-frame is encoded and stored in the buffer. Write all the data in the buffer from the start point to the last P-frame into the old video file. Then, starting from the I-frame in the buffer, begin writing the new video file.

[0103] Fast chunking mode: Immediately use the current cached data of the buffer as the old video file, and use the keyframe obtained from the next normal encoding as the starting keyframe of the next image group, thus obtaining the starting keyframe of the next image group.

[0104] The trigger condition for fast chunking mode is: T_continue > T_max; If the above triggering conditions are met, it indicates that waiting for the next I-frame would cause the old video file to last too long, severely impacting the user's video retrieval and viewing experience. To prioritize the temporal regularity of the video block, the encoder is instructed to generate a lightweight I-frame, which is then immediately written into the old video file as the end frame. Subsequently, the new video file begins with a standard keyframe or reference reset frame that is captured and encoded next.

[0105] Lightweight I-frames are I-frames with smaller data volumes. They can be obtained by reducing the coding quality or by coding a special I-frame that is intra-coded but not allowed to be referenced by subsequent frames.

[0106] The fast chunking mode ensures that the duration of each video file is controllable, avoiding the appearance of excessively long and meaningless video content at the end of a single video file due to waiting for I-frames, which would waste storage space and affect the user's viewing experience. It also solves the problem of spending too much time waiting for I-frames in the perfect chunking mode due to excessively long I-frame intervals or bitrate fluctuations.

[0107] Of course, in addition to the perfect slicing mode and fast slicing mode mentioned above, there may be other ways to generate keyframes, and this application does not limit them.

[0108] Example 2: The steps for generating the keyframe for the start of the next image group include the following steps S2321 to S2322.

[0109] Step S2321: Compare the current remaining space with the preset remaining space threshold to obtain the space comparison result.

[0110] Step S2322: Determine the matching video segmentation pattern based on the spatial comparison results, and execute the decision corresponding to the matching video segmentation pattern to generate the keyframe for the start of the next image group.

[0111] For example, the remaining space thresholds are sorted from largest to smallest, including the first threshold and the second threshold; based on the spatial comparison results, the following method one, method two, or method three is used to generate the keyframe for the start of the next image group.

[0112] Method 1: If the current remaining space is greater than the first threshold, compare the duration of the old video file with the maximum duration; determine whether to use the perfect segmentation mode or the fast segmentation mode based on the time comparison result. See the above embodiment for details.

[0113] When the remaining space is greater than the first threshold, the system determines whether to use the perfect segmentation mode or the fast segmentation mode to complete the video segmentation by comparing the duration T_continue of the old video file with the maximum duration T_max.

[0114] Method 2: If the current remaining space is less than the first threshold and greater than the second threshold, then increase the predicted cache space and / or decrease the maximum duration. Then compare the size relationship between the current remaining space and the predicted cache space, and compare the size relationship between the duration of the old video file and the maximum duration. Based on the comparison results, determine whether to adopt the perfect slicing mode or the fast slicing mode.

[0115] That is, when the remaining space is less than the first threshold but greater than the second threshold, the prediction buffer space is increased (e.g., the number of safety margin frames is increased), and / or the maximum duration is reduced. Then, based on the adjusted parameters, it is determined whether to perform video slicing and / or to select perfect slicing mode or fast slicing mode to complete video slicing.

[0116] Method 3: If the remaining space is less than the second threshold, reduce the encoding resolution and / or frame rate of the image data and adopt a fast chunking mode.

[0117] When the remaining space is less than the second threshold, the encoding resolution and / or frame rate are reduced directly, and video is immediately segmented. That is, the current cached data in the buffer is immediately used as the old video file, and the keyframe obtained by the next normal encoding is used as the keyframe for the start of the next image group.

[0118] The above embodiment of judging based on the current remaining space is only an illustrative example. In actual application scenarios, the used cache space of the cache can also be obtained for judgment. This application does not limit this.

[0119] For example, if the used cache space is less than 85%, the cache is considered to be in normal working condition. In this case, method one is used for video segmentation to ensure the segmentation effect. If 85% < used cache space < 95%, the cache is considered to be in a yellow alert state. In this case, method two is used for video segmentation, which can trigger segmentation more aggressively and / or tend to segment quickly in the early stages. If 95% < used cache space, the cache is considered to be in a red alert state. In this case, method three is used for video segmentation, which can enter the strongest degradation mode, temporarily reducing the encoding resolution or frame rate to generate lightweight I-frames as quickly as possible to complete video segmentation and ensure that no data is lost.

[0120] In one embodiment, in addition to adjusting the prediction cache space and / or maximum duration based on the current remaining space in Method 2 described above, the confidence level corresponding to the prediction position can also be obtained, and the prediction cache space and / or maximum duration can be adjusted based on the confidence level. For example, when the confidence level is low, the prediction cache space can be increased and / or the maximum duration can be decreased to trigger video segmentation earlier and avoid the impact of subsequent uncertain I-frames on the video segmentation effect.

[0121] In one embodiment, in order to improve the accuracy of the above-mentioned automatic video segmentation, the effect of each video segmentation can be detected, and the relevant parameters of video segmentation (such as the number of safety margin frames and / or the maximum duration) can be adjusted according to the detection results to achieve closed-loop optimization.

[0122] For example, by continuously detecting the deviation between the predicted position and the actual I-frame position of each video segment, and / or whether it leads to excessive use of buffer space, the video segmentation effect is obtained. The parameter adaptive engine adjusts the number of safety margin frames and / or the maximum duration based on the detected video segmentation effect, so that the system can adapt to different usage environments and achieve long-term personalized optimization.

[0123] Optionally, metadata such as video segmentation method (such as the perfect segmentation mode or fast segmentation mode mentioned above), first and last frame types, timestamps, and GOP information can be generated for each video file. At the same time, detailed system decision logs (such as predicted position, actual I-frame position, video segmentation triggering reason, etc.) can be recorded for subsequent analysis and self-learning.

[0124] In one embodiment, in order to reduce computational overhead, algorithm lightweighting (such as image downsampling before image analysis), hardware acceleration (such as using a digital signal processor (DSP) or neural processing unit (NPU) to handle computationally intensive tasks), and decision down-frequency (i.e., not executing the method of this application for every image frame, but executing the method of this application at a preset number of frame intervals) can ensure that the algorithm of this application runs in a low-power mode.

[0125] Please see Figure 4 , Figure 4 This is a flowchart illustrating a video storage method as shown in another exemplary embodiment of this application, such as... Figure 4 As shown, it includes: Step S401: Predict the position of the keyframe at the beginning of the next image group, and obtain the predicted position and the corresponding confidence score; Step S402: Sum the number of safety margin frames and the number of image frames between the current image frame position and the prediction position to obtain the prediction buffer space; Step S403: If the current remaining space is less than the predicted cache space, trigger video segmentation and calculate the duration T_continue of the old video file; Step S404: Is T_continue ≤ T_max? And / or, is the confidence of the predicted position greater than the threshold? If yes, proceed to step S405; otherwise, proceed to step S406. Step S405: Use the perfect slicing mode to slice the image, wait for the image data corresponding to the predicted position to be encoded as a keyframe, and obtain the keyframe for the beginning of the next image group. The old video file ends naturally with a P-frame. Step S406: Use the fast slicing mode to slice the video file, immediately generate a lightweight I-frame to end the old video file, and start the new video file with a standard keyframe that will be captured and encoded next. Step S407: Record the relevant parameters of the video segmentation to dynamically adjust T_max and the number of safety margin frames.

[0126] The video storage process is illustrated with an example of a specific application scenario: An AOV PTZ camera is deployed at the door of a home, the AOV acquisition interval is t=3 seconds, the total buffer capacity is C=1000 frames, the initial T_max=60 seconds, and the number of safety margin frames is 50 frames.

[0127] Scenario 1: Running in a static scenario Scene description: If the image changes little at night, it is determined that the current scene type is a static scene, and the GOP length is extended to 60 frames.

[0128] When the buffer has stored 900 frames, the next I-frame is predicted to be around frame 960. Calculations show N_p = 60, T_continue = 60 * 3 = 180s, and 180s > T_max(60s).

[0129] Decision: Immediately adopt the fast slicing mode for fast slicing: generate a lightweight I-frame to end the old video file, and the new video file starts with a standard I-frame, avoiding the old video file from generating a 3-minute meaningless video segment.

[0130] Scenario 2: Parameter tuning through self-learning Obtain log data of video segments recorded within a historical time period, and adjust relevant parameters by analyzing the log data.

[0131] Specifically, log data analysis revealed that frequent brief passages of people near residential entrances often caused T_continue to exceed the limit, leading to excessive use of the fast segmentation mode. Therefore, T_max was reduced from 60 seconds to 30 seconds, allowing the system to determine fast segmentation at an earlier stage, resulting in shorter video files that better suit the retrieval habits of security videos.

[0132] Scenario 3: Running in a dynamic scenario Scene description: When a visitor is detected lingering at the door at night, it is determined that the current scene type is a dynamic scene, and the GOP length is shortened to 20 frames.

[0133] If the next I-frame is predicted to be 10 frames later, then T_continue = 10 * 3 = 30 seconds. At this time, T_continue is equal to T_max(30 seconds).

[0134] Decision: Use perfect slicing mode for perfect slicing: wait 30 seconds, the old video file will end naturally with 10 P frames, and the new video file will be started with the next I frame. At this time, the storage efficiency reaches the optimal level.

[0135] Scenario 4: Handling extreme situations Scene description: A persistent storm caused raindrops and swaying leaves to move in complex motion for an extended period in the footage, preventing the old video file from being successfully segmented. The cache slowly increased, eventually reaching the yellow warning line of the cache (used cache space is higher than 85%).

[0136] Decision: Method 2 was adopted for segmentation judgment, and video segmentation was performed quickly, successfully releasing cache pressure before the cache reached the red alert line.

[0137] As can be seen from the above examples, this application can predict the position of the keyframe at the start of the next image group, thus sensing the timing of video segmentation in advance. This allows for precise video segmentation at the keyframe position, improving storage efficiency. It can also adaptively adjust to different situations and perform closed-loop parameter optimization to continuously adapt to the actual deployment environment and improve the accuracy of the solution execution.

[0138] Additionally, this application also protects a video storage system; please refer to [link to relevant documentation]. Figure 5 , Figure 5 This is a schematic diagram of a video storage system illustrating an exemplary embodiment of this application, such as... Figure 5 As shown, it includes an adaptive encoder, a cache management module, a GOP management and prediction module, a video segmentation decision-maker, a parameter adaptive engine in the video segmentation decision-maker, and a data recording module.

[0139] The adaptive encoder is used to encode the acquired image data to obtain an encoded bitstream, which is then sent to the cache management module. The module can generate different types of keyframes according to instructions, such as standard I-frames and low-complexity keyframes (such as reference reset frames). In emergencies, it can also support reducing the resolution / frame rate to quickly generate lightweight I-frames.

[0140] The cache management module is used to manage the read and write operations of the cache (writing the segmented video file to persistent storage), read and write the encoded bitstream, and monitor the cache's operating information (such as the number of frames stored, remaining space, etc.).

[0141] The GOP management and prediction module is used to flexibly adjust the GOP length and predict the position of the keyframe at the beginning of the next image group based on the buffer's running information.

[0142] The video segmentation decision-maker determines the specific method for video segmentation (such as selecting either perfect segmentation mode or fast segmentation mode) and sends corresponding instructions to the adaptive encoder and cache management module.

[0143] The parameter adaptive engine dynamically adjusts key parameters (such as T_max, number of safety margin frames, etc.).

[0144] The data recording module is used to generate metadata for each video file, including video segmentation method (such as the perfect segmentation mode or fast segmentation mode mentioned above), first and last frame types, timestamps, GOP information, etc. At the same time, it records detailed system decision logs (such as predicted position, actual I-frame position, video segmentation triggering reasons, etc.). These data are stored in persistent storage for subsequent analysis and self-learning.

[0145] Figure 6 This is a block diagram illustrating a video storage device according to an exemplary embodiment of this application. Figure 6 As shown, the exemplary video storage device 600 includes: The caching and prediction module 610 is used to encode the real-time acquired image data and store it in the cache, and predict the position of the key frame at the beginning of the next image group to obtain the predicted position. The cache space determination module 620 is used to calculate the predicted cache space based on the cache space required for image frames between the current image frame position and the predicted position, and to obtain the current remaining space of the cache. The video slicing module 630 is used to write data of the new video file to the buffer starting from the key frame encoding of the next picture group, in response to the completion of the key frame encoding of the next picture group if the size relationship between the current remaining space and the predicted buffer space meets the preset conditions, and to transfer the buffered data before the key frame to the persistent storage as the data of the old video file.

[0146] It should be noted that the video storage device and the video storage method provided in the above embodiments belong to the same concept. The specific ways in which each module and unit performs operations have been described in detail in the method embodiments, and will not be repeated here. In practical applications, the video storage device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation.

[0147] Please see Figure 7 , Figure 7 This is a schematic diagram illustrating the structure of an electronic device according to an exemplary embodiment of this application. The electronic device 700 includes a memory 701 and a processor 702. The processor 702 executes program instructions stored in the memory 701 to implement the steps in any of the video storage method embodiments described above. In a specific implementation scenario, the electronic device 700 may include, but is not limited to, a microcomputer or a server. Furthermore, the electronic device 700 may also include mobile devices such as laptops and tablets, without limitation.

[0148] Specifically, processor 702 controls itself and memory 701 to implement the steps in any of the video storage method embodiments described above. Processor 702 can also be referred to as a Central Processing Unit (CPU). Processor 702 may be an integrated circuit chip with signal processing capabilities. Processor 702 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 702 can be implemented using integrated circuit chips.

[0149] Please see Figure 8 , Figure 8 This is a schematic diagram illustrating the structure of a computer-readable storage medium according to an exemplary embodiment of this application. The computer-readable storage medium 800 stores program instructions 810 that can be executed by a processor. The program instructions 810 are used to implement the steps in any of the above-described video storage method embodiments.

[0150] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0151] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0152] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0153] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A video storage method characterized by comprising: The method includes: The real-time acquired image data is encoded and stored in a buffer, and the position of the keyframe at the beginning of the next image group is predicted to obtain the predicted position. Based on the cache space required for image frames between the current image frame position and the predicted position, the predicted cache space is calculated, and the current remaining space of the cache is obtained. If the size relationship between the current remaining space and the predicted cache space meets a preset condition, then in response to the completion of the keyframe encoding at the beginning of the next image group, data for the new video file is written to the cache starting from the keyframe, and the cached data previously stored in the cache is transferred to the persistent storage as data for the old video file.

2. The method of claim 1, wherein, The process of predicting the position of the keyframe at the start of the next image group, and obtaining the predicted position, includes: The positions of keyframes in the cached image group are statistically analyzed to obtain the keyframe position patterns; and / or, mutation prediction is performed based on the image content of the real-time acquired image data to obtain mutation prediction results. Based on the keyframe position pattern and / or the mutation prediction result, the position of the keyframe at the beginning of the next image group is predicted to obtain the predicted position.

3. The method of claim 1, wherein, The prediction cache space is calculated based on the cache space required for image frames between the current image frame position and the predicted position, including: Calculate the number of image frames between the current image frame position and the predicted position, and obtain the preset safety margin frame number; The prediction buffer space is obtained by summing the number of image frames and the number of safety margin frames.

4. The method of claim 1, wherein, The steps to generate the keyframe for the start of the next image group include: Calculate the number of image frames between the current image frame position and the predicted position; Based on the number of image frames and the frame interval time, the duration of the old video file is calculated, and the preset maximum duration is obtained. By comparing the duration of the old video file with the maximum duration, a time comparison result is obtained; Based on the time comparison results, a matching video segmentation pattern is determined, and the decision corresponding to the matching video segmentation pattern is executed to generate the keyframe for the start of the next image group.

5. The method of claim 4, wherein, The step of determining the matching video segmentation pattern based on the time comparison result and executing the decision corresponding to the matching video segmentation pattern to generate the keyframe for the start of the next image group includes: If the duration of the old video file is less than or equal to the maximum duration, then the perfect segmentation mode is adopted. The decision of the perfect segmentation mode includes: waiting for the image data corresponding to the predicted position to be encoded as a keyframe to obtain the keyframe for the start of the next image group. If the duration of the old video file is greater than the maximum duration, a fast slicing mode is adopted. The decision of the fast slicing mode includes: immediately using the current cached data of the buffer as the old video file, and using the keyframe obtained by the next normal encoding as the keyframe of the next image group.

6. The method of claim 1, wherein, The steps to generate the keyframe for the start of the next image group include: The space comparison result is obtained by comparing the current remaining space with the preset remaining space threshold. Based on the spatial comparison results, a matching video segmentation pattern is determined, and the decision corresponding to the matching video segmentation pattern is executed to generate the keyframe for the start of the next image group.

7. The method of claim 6, wherein, The video slicing modes include perfect slicing mode and fast slicing mode, and the remaining space thresholds are sorted from largest to smallest, including a first threshold and a second threshold; determining the matching video slicing mode based on the space comparison results includes: If the current remaining space is greater than the first threshold, then compare the duration of the old video file with the maximum duration; based on the time comparison result, determine whether to use the perfect slicing mode or the fast slicing mode. If the current remaining space is less than the first threshold and greater than the second threshold, then after increasing the predicted cache space and / or decreasing the maximum duration, the size relationship between the current remaining space and the predicted cache space, and the size relationship between the duration of the old video file and the maximum duration are compared, and based on the comparison results, it is determined whether to adopt the perfect slicing mode or the fast slicing mode. If the current remaining space is less than the second threshold, the encoding resolution and / or frame rate of the image data are reduced, and a fast chunking mode is adopted.

8. The method of claim 1, wherein, The method further includes: The cache is divided into: an old block area, a transition area, and a new block preparation area; The old block area is used to store image frames that will be written to the old video file; The transition region is used to store subsequent image frames of the current image group, and the size of the transition region is related to the prediction cache space. The new block preparation area is the reserved space required for the starting keyframe and subsequent image frames of the new video file.

9. The method according to any one of claims 1 to 8, characterized in that, The method further includes: Perform scene change analysis on real-time acquired image data and classify the current scene type; If the current scene type is a static scene, then extend the image group length; If the current scene type is a dynamic scene, then shorten the image group length.

10. An electronic device, comprising: The electronic device includes a memory and a processor, the processor being configured to execute program instructions stored in the memory to implement the steps of the method as described in any one of claims 1-9.

11. A computer readable storage medium characterized by, The computer-readable storage medium stores program instructions that can be executed by a processor to implement the steps of the method as described in any one of claims 1-9.

Images