Pre-recorded lossless video storage method and system

Abstract

The invention relates to a video pre-recording processing technology, and discloses a pre-recorded lossless video storage method and system, and the method comprises the steps: arranging each frame of image forming a video according to a bit sequence and / or a time sequence in a video pre-recording process, and forming different image sequences; in the image sequence, arranging pixel contents forming each frame of image according to a bit sequence and / or a time sequence to form different pixel bit sequences, and carrying out complete frame storage on the acquired first frame of image according to the pixel contents of the frame of image; storage of the compressed frame: obtaining a pixel difference value of a position corresponding to the nearest complete frame according to the content of each pixel of the compressed frame in the image sequence, and storing the pixel difference value of the frame; and when the storage space of the compressed frame reaches a set threshold value, the next frame is stored as a complete frame. According to the invention, in the pre-recording state, the data volume stored in each frame is reduced by utilizing the continuous static state of the picture and no change of the picture, so that the storage space is reduced.

Description

Technical Field

[0001] The invention relates to a video pre-recording processing technology, and in particular to a lossless video storage method and system under pre-recording. Background Art

[0002] In order to save power when the current low-power camera is in sleep mode, when there is no object motion in the scene or no object of interest exists, the main control (ISP) stops working and pre-records each frame of raw format data by adding a psram method on the CMOS image sensor (hereinafter referred to as sensor). When a person is detected or the pre-recording time is up (the pre-recording storage space is full), the main control is awakened.

[0003] As in prior art 1: CN201180034171.0, a method and device for processing a video frame includes: dividing a video frame into a plurality of pixel blocks, each pixel block including a reference pixel; calculating a difference between a reference pixel and a pixel adjacent to the reference pixel; converting the calculated difference into a value belonging to a bit depth range; and performing entropy coding by using the converted difference as a symbol. Summary of the invention

[0004] The present invention aims at the problem that when a current low-power camera pre-records each frame through a sensor end in the prior art, as the number of pre-recorded frames increases, the larger the external PSRAM space is, the higher the cost is; at the same time, the SENSOR end is a photoelectric conversion sensor, and the data processing and computing memory are limited; in addition, after the main control is awakened, the amount of multi-frame data acquired at one time is large, which increases the transmission cost. The present invention provides a lossless video storage method and system under pre-recording.

[0005] In order to solve the above technical problems, the present invention is solved by the following technical solutions: A method for pre-recording lossless video storage, wherein when a sensor pre-records a video, the storage of the video includes: Storage of complete frames: During the video pre-recording process, each frame of the video is arranged according to a bit sequence and / or a time sequence to form different image sequences; wherein the total number of frames of the image sequence is N frames; in the image sequence, the pixel contents of each frame of the image are arranged according to a bit sequence and / or a time sequence to form different pixel bit sequences, and the first frame of the image obtained is stored as a complete frame according to the pixel contents of the frame of the image; Storage of compressed frames: In the image sequence, the pixel difference between the corresponding position of the compressed frame and the nearest complete frame is obtained according to the content of each pixel of the compressed frame, and the pixel difference of the frame is stored. When the storage space of the current compressed storage frame reaches the set threshold, the next frame is stored as a complete frame.

[0006] Preferably, the storage of compressed frames comprises: The pixel value at the first pixel position of the complete frame image in the image sequence is set as the pixel value of the complete frame, the pixel value at the same pixel position of the next frame image adjacent to the complete frame image in the image sequence is set as the pixel value of the next frame, and the pixel value of the complete frame is subtracted from the pixel value of the next frame to obtain the pixel difference value in the image sequence; The difference value of the pixel is then stored in the corresponding pixel position in the next frame of image.

[0007] Preferably, storing according to pixel differences in the image sequence comprises: Obtaining the largest pixel difference among the pixel differences, and determining the data length of the largest pixel difference; The obtained pixel difference value of each position is converted into pixel content of the data length and stored in the pixel sequence of the image sequence.

[0008] Preferably, forming different pixel sequences further comprises grouping a frame of images to obtain a grouped image sequence.

[0009] Preferably, storing the grouped image sequence includes: Obtain pixel difference values ​​in each set of image sequences; and obtaining a maximum pixel difference value from the pixel difference values ​​of each group, and determining a storage length of image pixels in each group of image sequences according to the maximum pixel difference value; The pixel difference values ​​in the image sequence of the group are converted into pixel contents of the data length and stored in the pixel sequence of the group.

[0010] Preferably, the storage of compressed frames also includes determining the total length of the compressed frame storage. When the total length of the compressed frame storage reaches a set storage length threshold, the next frame is stored as a complete frame, otherwise the compressed frame storage continues.

[0011] In order to solve the above technical problems, the present invention also provides a lossless video storage system under pre-recording. When the sensor pre-records the video, the storage of the video includes: Full frame storage module: During the video pre-recording process, each frame of the video is arranged according to a bit sequence and / or a time sequence to form different image sequences; wherein the total number of frames of the image sequence is N frames; in the image sequence, the pixel contents of each frame of the image are arranged according to a bit sequence and / or a time sequence to form different pixel bit sequences, and the first frame of the image obtained is stored as a complete frame according to the pixel contents of the frame of the image; Compressed frame storage module: In the image sequence, the pixel difference between the corresponding position of the compressed frame and the nearest complete frame is obtained according to the content of each pixel of the compressed frame, and the pixel difference of the frame is stored. When the storage space of the current compressed storage frame reaches the set threshold, the next frame is stored as a complete frame.

[0012] The present invention has significant technical effects due to the adoption of the above technical solution: The present invention can perform lossless compression of video in a low-power pre-recording mode; The present invention adopts a pixel difference method to dynamically adjust the storage bits occupied by different pixel groups during storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a flow chart of the present invention.

[0014] Figure 2 It is a schematic diagram of storing a complete frame with a pixel bit depth of 10 bits according to the present invention.

[0015] Figure 3 It is a schematic diagram of compressed frame storage of the present invention.

[0016] Figure 4 This is the process of obtaining recorded frames after the main control is awakened in the present invention.

[0017] Figure 5 It is a schematic diagram of the pre-recording process of the present invention. DETAILED DESCRIPTION

[0018] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0019] Example 1

[0020] A method for pre-recording lossless video storage, wherein when a sensor pre-records a video, the storage of the video includes: Frame sequence storage: During the video pre-recording process, each frame of the video is arranged in a bit sequence and / or a time sequence to form different image sequences; wherein the total number of frames in the image sequence is N frames; in the image sequence, the pixel contents of each frame are arranged in a bit sequence and / or a time sequence to form different pixel bit sequences.

[0021] The SENSOR uses rolling shutter exposure, that is, the exposure start and end time of each row are the same, and the start time of the next row is a certain time later than the previous row. After each row of pixels is exposed and gain amplified (according to the gain configuration issued by the main control AE algorithm), before the next row of pixels arrives, it is decided whether to store it directly in the external PSRAM or store it in the external PSRAM after pixel processing, depending on whether the current frame is a complete frame storage or a compressed frame storage.

[0022] Storage of complete frames: When receiving the signal from the master control to start pre-recording, the first frame of the video pre-recording is stored as a complete frame, and the pixel contents constituting the image in the frame are directly stored in the external PSRAM in row order and bit order, and the offset of each row stored in the PSRAM is temporarily stored in the SENSOR cache (such as a register); in the subsequent frame image sequence, each frame is stored as a compressed frame, and when the storage space of a compressed frame reaches the specified threshold, it is marked that the next frame is stored as a complete frame; Storage of compressed frames: In the image sequence, the pixel difference between the corresponding position of the compressed frame and the nearest complete frame is obtained according to the content of each pixel of the compressed frame, and the pixel difference of the frame is stored. When the storage space of the current compressed storage frame reaches the set threshold, the next frame is stored as a complete frame.

[0023] For example, an image has N frames, each frame has m rows and n columns; for a pixel in the image, （i，j） ; Where i is 0, 1…m; j is 0, 1…n; For the first frame image, p0 （i，j） ; then the corresponding complete frame closest to the first frame is p1 （i，j） ; Figure 5 In the process of pre-recording the video, after the current row of pixels is exposed and converted, the row of pixels corresponding to the nearest complete frame is obtained from the PSRAM; the difference calculation is performed on the new row of pixels to generate a variable length pixel format and write it into the PSRAM.

[0024] Storage of compressed frames includes: The pixel value at the first pixel position of the complete frame image in the image sequence is set as the pixel value of the complete frame, the pixel value at the same pixel position of the next frame image adjacent to the complete frame image in the image sequence is set as the pixel value of the next frame, and the pixel value of the complete frame is subtracted from the pixel value of the next frame to obtain the pixel difference value in the image sequence; The difference value of the pixel is then stored in the corresponding pixel position in the next frame of image.

[0025] For example, the pixel in the first row is p0 （0，j） ; then the first row of pixels of the most recent complete frame is p1 （0，j） ; For the difference h （0，j） ; h （0，0） =p0 （0，0） -p1 （0，0） ; h （0，1） =p0 （0，1） -p1 （0，1） ; ……; h （0，j） =p0 （0，j） -p1 （0，j） ; The calculated h （0，j） Convert to the pixel format corresponding to the original image pixel value, such as p0 （0，0） The pixel is 420, which is stored as "110100100"; p1 （0，0）） The pixel is 422, which is stored as "110100110"; the calculated pixel difference is 2; store the difference 2 in p1 （0，j） The storage mode is "000000010"; write "000000010" into PSRAM.

[0026] In the Bayer format of the sensor's raw data, the difference between most of the same pixels in the previous and next frames is very small. For example, if the same pixel data in the previous frame is 450, the next pixel may be 448 or 452. Using the above characteristics, a complete raw data frame is stored first, and the pixel data of the subsequent frames is subtracted from the same pixel value of the previous complete raw data frame, and only the difference is stored; the next pixel is stored as +2 or -2. As the value becomes smaller, the required storage bit changes from a complete 10bit / 12bit to 0bit, 1bit or 2bit, which can reduce a lot of storage space, and the maximum space saved can reach 1 / 5 of the original.

[0027] Example 2

[0028] Based on the first embodiment, the present embodiment stores pixel differences in an image sequence, including: Obtain the maximum pixel difference value among the pixel differences in each row of grouped pixels, and determine the storage length occupied by each pixel difference value of the group in a signed data mode, and the number of bits occupied; The obtained pixel difference value of each position is converted into pixel content of the data length and stored in the pixel sequence of the image sequence.

[0029] For each pixel h in the first row and the first group （0，0）， h （0，1），…… h （0，k） Difference: h （0，0） =p0 （0，0） -p1 （0，0） ; h （0，1） =p0 （0，1） -p1 （0，1） ; ……; h （0，k） =p0 （0，k） -p1（0，k） ; where hmax=max(h （0，0） ,h （0，1） …h （0，k） ); According to the calculated maximum difference, determine the data length of hmax as L; and set h （0，0）， h （0，1）…… h （0，k） The pixel difference of other digits is converted into the data length L; For example: (h （0，0） ,h （0，1） …h （0，k） ) = (0, 2, 4...4); then the maximum difference, hmax, corresponds to a data length of 3 bits, h （0，0）， h （0，1）…… h （0，k） All are converted into 3-bit data for storage. （0，0） Convert the binary "000" to store; （0，1） Convert the binary "010" to store; （0，2） The conversion bit is stored as binary "100"; similarly h （0，k） The converted bit is stored as binary "100".

[0030] Example 3

[0031] Based on the above embodiment, the forming of different pixel sequences in this embodiment further includes grouping a frame of images to obtain a grouped image sequence.

[0032] The grouped image sequence is stored including: Obtain pixel differences in each group of image sequences; obtain the maximum pixel difference from the pixel differences in each group, and determine the storage length of image pixels in each group of image sequences based on the maximum pixel difference; convert the pixel differences in the image sequence of the group into pixel contents of the data length, and store them in the pixel sequence of the group.

[0033] For example, the pixel in the first row is p0 （0，j） ; Group the pixels in the first row into a groups, each group has k pixels, and a*k=j. The first row of pixels in the most recent complete frame is p1 （0，j） ; Similarly, the first row of pixels of the most recent complete frame is grouped into a groups, each group of pixel data is k, and a*k=j; For this, the difference between each group is the first group h （0，k） is (h （0，1）， h （0，2） ……h （0，k） ); The second group h （0，2k） is (h （0，k+1）， h （0，k+2） ……,h（0，2k） ); Group a h （0，ak） is (h （0，ak-k+1）， h （0，ak-k+1） ……,h （0，ak） ); Determine h （0，k） ,h （0，2k） …h （0，ak） The maximum pixel difference hmax of the group （0，k） , hmax （0，2k） …hmax （0，ak） ; For example: (h （0，0） ,h （0，1）， …,h （0，k） ) = (0, 2, 4...4); then the maximum difference, hmax, corresponds to a data length of 3 bits, h （0，0）， h （0，1）…… h （0，k） All are converted into 3-bit data for storage. （0，0） Convert the binary "000" to store; （0，1） Convert the binary "010" to store; （0，2） The conversion bit is stored as binary "100"; similarly h （0，k） The converted bit is stored as binary "100".

[0034] The above divides a row of pixels into a group, such as 2 million pixels (row width is 1080), which can be divided into 20 groups, a=20; each group has 54 pixels. After a row of pixels is exposed, gain amplified, and ADC converted, the pixel values ​​of the corresponding row of the nearest complete frame in the storage space are subtracted to obtain the difference of each pixel; the number of bits occupied by the maximum difference of pixels in each group is analyzed to obtain the space occupied by each pixel in the group; the number of bits occupied by each group of pixels and the difference of each pixel are stored in the corresponding space position, and the storage format is as follows Figure 3 The schematic diagram of compressed frame storage is shown.

[0035] Example 4

[0036] Based on the above embodiment, the storage of compressed frames in this embodiment further includes the determination of the total length of compressed frame storage. When the total length of compressed frame storage reaches the set storage length threshold, the next frame is stored as a complete frame, otherwise the compressed frame storage continues. When the total length of compressed frame storage is large (such as a 2-megapixel sensor with a bit depth of 10 bits per pixel, a complete frame requires about 2.5MB of storage space. When the total length of compressed frame storage exceeds 1MB and reaches the set storage length threshold, it indicates that the static state of the current picture has changed, and the next frame storage can be stored as a complete frame.

[0037] After the master end wakes up, it obtains all the stored frame data and restores them frame by frame. For compressed frames, the difference stored for each pixel is added to the corresponding pixel value of the nearest complete frame, and the actual data is restored losslessly.

[0038] Figure 3 In the compressed frame storage pixel group, the number of bits per pixel is fixed to 4 bits, and its value indicates the number of bits occupied by each pixel in the group. If it is 0, it indicates that each pixel in the group does not need to occupy space. When it is other values, it indicates that the stored difference occupies space. The space occupied by the largest pixel difference in each group conforms to binary. For example, the maximum value is 2, indicating that 2 bits of space are occupied.

[0039] When the main control unit sends a wake-up signal, the sensor stops pre-recording and starts marking each stored frame. It determines whether it is a complete frame storage or a compressed frame storage based on the mark. For the storage of compressed frames, the difference data is taken row by row and group by group, and the corresponding pixel values ​​of the complete frame are added to restore the original pixel data. For complete frame data, the complete frame data is directly obtained.

[0040] Example 5

[0041] Based on the above embodiment, this embodiment is a lossless video storage system under pre-recording. When the sensor pre-records the video, the storage of the video includes: Full frame storage module: During the video pre-recording process, each frame of the video is arranged according to a bit sequence and / or a time sequence to form different image sequences; wherein the total number of frames of the image sequence is N frames; in the image sequence, the pixel contents of each frame of the image are arranged according to a bit sequence and / or a time sequence to form different pixel bit sequences, and the first frame of the image obtained is stored as a complete frame according to the pixel contents of the frame of the image; Compressed frame storage module: In the image sequence, the pixel difference between the corresponding position of the compressed frame and the nearest complete frame is obtained according to the content of each pixel of the compressed frame, and the pixel difference of the frame is stored. When the storage space of the current compressed storage frame reaches the set threshold, the next frame is stored as a complete frame.

Claims

1. A method for storing lossless pre-recorded videos, wherein when a sensor pre-records a video, the storage of the video includes: Storage of complete frames: During the video pre-recording process, each frame of the video is arranged according to a bit sequence and / or a time sequence to form different image sequences; wherein the total number of frames of the image sequence is N frames; in the image sequence, the pixel contents of each frame of the image are arranged according to a bit sequence and / or a time sequence to form different pixel bit sequences, and the first frame of the image obtained is stored as a complete frame according to the pixel contents of the frame of the image; Storage of compressed frames: In the image sequence, the pixel difference between the corresponding position of the compressed frame and the nearest complete frame is obtained according to the content of each pixel of the compressed frame, and the pixel difference of the frame is stored; when the storage space of the compressed storage frame reaches the set threshold, the next frame is stored as a complete frame.

2. The method for storing pre-recorded lossless video according to claim 1, characterized in that: Storage of compressed frames includes: The pixel value at the first pixel position of the complete frame image in the image sequence is set as the pixel value of the complete frame, the pixel value at the same pixel position of the next frame image adjacent to the complete frame image in the image sequence is set as the pixel value of the next frame, and the pixel value of the complete frame is subtracted from the pixel value of the next frame to obtain the pixel difference value in the image sequence; The difference value of the pixel is then stored in the corresponding pixel position in the next frame of image.

3. The method for storing pre-recorded lossless video according to claim 2, characterized in that: Storage based on pixel differences in an image sequence includes: Obtaining the largest pixel difference among the pixel differences, and determining the data length of the largest pixel difference; The obtained pixel difference value of each position is converted into pixel content of the data length and stored in the pixel sequence of the image sequence.

4. The method for storing lossless pre-recorded video according to claim 2, characterized in that: Forming different pixel sequences also includes grouping a frame of images to obtain a grouped image sequence.

5. The method for storing pre-recorded lossless video according to claim 4, characterized in that: The grouped image sequence is stored including: Obtain pixel difference values ​​in each set of image sequences; and obtaining a maximum pixel difference value from the pixel difference values ​​of each group, and determining a storage length of image pixels in each group of image sequences according to the maximum pixel difference value; The pixel difference values ​​in the image sequence of the group are converted into pixel contents of the data length and stored in the pixel sequence of the group.

6. The method for storing pre-recorded lossless video according to claim 1, characterized in that: The storage of compressed frames also includes the determination of the total length of the compressed frame storage. When the total length of the compressed frame storage reaches the set storage length threshold, the next frame is stored as a complete frame, otherwise the compressed frame storage continues.

7. A pre-recorded lossless video storage system, characterized in that: When the sensor pre-records a video, the video storage includes: Full frame storage module: During the video pre-recording process, each frame of the video is arranged according to a bit sequence and / or a time sequence to form different image sequences; wherein the total number of frames of the image sequence is N frames; in the image sequence, the pixel contents of each frame of the image are arranged according to a bit sequence and / or a time sequence to form different pixel bit sequences, and the first frame of the image obtained is stored as a complete frame according to the pixel contents of the frame of the image; Compressed frame storage module: In the image sequence, the pixel difference between the corresponding position of the compressed frame and the nearest complete frame is obtained according to the content of each pixel of the compressed frame, and the pixel difference of the frame is stored; when the storage space of the current compressed storage frame reaches the set threshold, the next frame is stored as a complete frame.

Images