A method and system for storing pre-recorded lossless videos

Abstract

The present invention relates to video pre-recording processing technology, and discloses a lossless video storage method and system for pre-recording, which includes: during the video pre-recording process, arranging each frame of the image constituting the video according to bit order and / or time sequence to form different image sequences; in the image sequences, arranging the pixel contents of each frame of the image according to bit order and / or time sequence to form different pixel bit orders, and storing the obtained first frame of the image as a complete frame according to the pixel content of this frame of the image; storing compressed frames: obtaining the pixel difference between the corresponding position of the nearest complete frame according to the pixel content of each pixel of the compressed frame in the image sequences, and storing this frame of pixel difference; when the storage space of the compressed frame reaches the set threshold, the next frame is stored as a complete frame. By using the state where the picture remains stationary and there is no change in the pre-recording state, the present invention reduces the amount of data stored in each frame to achieve reduction of storage space.

Description

Technical Field

[0001] The present invention relates to video pre-recording processing technology, and particularly to a lossless video storage method and system for pre-recording. Background Art

[0002] When the current low-power camera is in the sleep state, in order to save power, when there is no object movement in the scene or no object of interest exists, the main control (ISP) stops working, and the psram is added at the CMOS image sensor (hereinafter referred to as sensor) end to pre-record each frame of raw format data. When a person appears or after the pre-recording time arrives (the pre-recording storage space is full), the main control is awakened.

[0003] As in the prior art 1: CN201180034171.0, a method and device for processing video frames, the method includes: dividing a video frame into a plurality of pixel blocks, each pixel block including a reference pixel; calculating a difference between the reference pixel and a pixel adjacent to the reference pixel; converting the calculated difference into a value belonging to a bit depth range; and performing entropy coding by using the converted difference as a symbol. Summary of the Invention

[0004] In view of the problems in the prior art that when the current low-power camera pre-records each frame of the picture through the sensor end, as the number of pre-recorded frames increases, the larger the external psram space is, the higher the cost is. At the same time, the SENSOR end is an optoelectronic conversion sensor, with limited data processing and computing memory. In addition, after the main control is awakened, the large amount of multi-frame data obtained at one time increases the transmission cost, the present invention provides a lossless video storage method and system for pre-recording.

[0005] In order to solve the above technical problems, the present invention is solved by the following technical solutions:

[0006] A lossless video storage method for pre-recording, when the sensor pre-records the video, the storage of the video includes:

[0007] Storage of complete frames:

[0008] During the video pre-recording process, each frame of the image forming the video is arranged according to bit order and / or time sequence to form different image sequences; wherein, the total number of frames of the image sequence is N frames; in the image sequence, the pixel contents of each frame of the image are arranged according to bit order and / or time sequence to form different pixel bit orders, and the first frame of the image obtained is stored as a complete frame according to the pixel content of the frame image.

[0009] Storage of compressed frames:

[0010] In the image sequence, the pixel difference from the corresponding position of the nearest complete frame is obtained according to the content per pixel of the compressed frame, and the pixel difference of this frame is stored. When the storage space of the current compressed storage frame reaches the set threshold, the next frame is stored as a complete frame.

[0011] Preferably, the storage of the compressed frame includes:

[0012] Set the pixel value at the first pixel position of the complete frame image in the image sequence to the pixel value of the complete frame, set the pixel value at the same pixel ordinal position of the next frame adjacent to the complete frame image in the image sequence to the pixel value of the next frame, subtract the pixel value of the complete frame from the pixel value of the next frame to obtain the pixel difference in the image sequence;

[0013] Then store the difference of this pixel at the corresponding pixel position in the next frame image.

[0014] Preferably, the storage based on the pixel difference in the image sequence includes:

[0015] Obtain the largest pixel difference among the pixel differences and determine the data length of the largest pixel difference;

[0016] Convert the pixel difference at each obtained position into the pixel content of this data length and store it in the pixel ordinal position of the image sequence.

[0017] Preferably, forming different pixel ordinal positions further includes grouping a frame of image to obtain a grouped image sequence.

[0018] Preferably, the storage of the grouped image sequence includes:

[0019] Obtain the pixel difference in each group of image sequences;

[0020] And obtain the largest pixel difference from the pixel differences in each group, and determine the storage length of the image pixels in each group of image sequences according to the largest pixel difference;

[0021] Convert the pixel difference in the image sequence of this group into the pixel content of this data length and store it in the pixel ordinal position of this group.

[0022] Preferably, the storage of the compressed frame further includes judging the total length of the compressed frame storage. When the total length of the compressed frame storage reaches the set storage length threshold, the next frame is stored as a complete frame, otherwise, the compressed frame storage continues.

[0023] To solve the above technical problems, the present invention also provides a lossless video storage system for pre-recording. When the sensor pre-records a video, the storage of the video includes:

[0024] Complete frame storage module:

[0025] During the video pre-recording process, each frame image that makes up the video is arranged according to bit order and / or time sequence to form different image sequences; among them, the total number of frames in the image sequence is N frames; in the image sequence, the pixel contents that make up each frame image are arranged according to bit order and / or time sequence to form different pixel bit orders, and the first frame image obtained is stored as a complete frame according to the pixel content of this frame image;

[0026] Compressed frame storage module:

[0027] In the image sequence, the pixel difference between the corresponding position of the current compressed frame and the nearest complete frame is obtained according to the pixel content of each pixel, and this frame of pixel difference is stored. When the storage space of the current compressed storage frame reaches the set threshold, the next frame is stored as a complete frame.

[0028] Due to the adoption of the above technical solutions, the present invention has significant technical effects:

[0029] The present invention can perform lossless compression of videos in a low-power pre-recording mode;

[0030] By adopting the method of pixel difference, the present invention dynamically adjusts the storage bits occupied by different pixel groups during storage. Brief Description of the Drawings

[0031] Figure 1 is the flow chart of the present invention.

[0032] Figure 2 is the schematic diagram of the complete frame storage with a pixel bit depth of 10 bits of the present invention.

[0033] Figure 3 is the schematic diagram of the compressed frame storage of the present invention.

[0034] Figure 4 is the process of obtaining the recorded frame after the main control wakes up in the present invention.

[0035] Figure 5 is the schematic diagram of the pre-recording process of the present invention. Detailed Description of the Invention

[0036] The present invention will be further described in detail below with reference to the drawings and embodiments.

[0037] Embodiment 1

[0038] A lossless video storage method under pre-recording. When the sensor pre-records a video, the storage of the video includes:

[0039] Storage of frame order:

[0040] During video pre-recording, each frame image that makes up the video is arranged according to bit order and / or time sequence to form different image sequences; among them, the total number of frames in the image sequence is N frames; in the image sequence, the pixel content that makes up each frame image is arranged according to bit order and / or time sequence to form different pixel bit orders.

[0041] The SENSOR is in the rolling shutter exposure mode, that is, the start and end times of exposure for each row are the same, and the start time of the next row is a certain time later than the previous row. After the exposure of each row of pixels is completed and the pixels of that row are gain-amplified (according to the gain configuration sent by the main control AE algorithm), before the next row of pixels arrives, it is decided whether to directly store it in the external PSRAM or to store it in the external PSRAM after pixel processing according to whether the current frame is stored as a complete frame or a compressed frame.

[0042] Storage of complete frames:

[0043] When receiving the start of pre-recording from the main control, the first frame of video pre-recording is stored as a complete frame. The pixel content that makes up the intra-frame image is directly stored in the external PSRAM according to row order and bit order, and the offset of each row stored in the PSRAM is temporarily stored in the SENSOR buffer (such as a register); in the subsequent frame image sequence, each frame is stored as a compressed frame. When the storage space of a compressed frame reaches the specified threshold, it is indicated that the next frame is stored as a complete frame.

[0044] Storage of compressed frames:

[0045] In the image sequence, the pixel difference corresponding to the nearest complete frame is obtained for each pixel content of the compressed frame, and the pixel difference of this frame is stored. When the storage space of the current compressed storage frame reaches the set threshold, the next frame is stored as a complete frame.

[0046] For example, for an image with N frames, each frame image has m rows and n columns; for the pixels in the image as p （i，j） ; where i is 0, 1...m; j is 0, 1...n;

[0047] For the first frame image as p0 （i，j） ; then the nearest complete frame corresponding to it is p1 （i，j） ;

[0048] Figure 5 Among them, during the video pre-recording process, after the exposure and gain conversion of the current row of pixels are completed, the row pixels corresponding to the nearest complete frame are obtained from the PSRAM; the difference calculation is performed on the new row of pixels, thereby generating a variable-length pixel format and writing it into the PSRAM.

[0049] The storage of compressed frames includes:

[0050] Set the pixel value at the first pixel position of the complete frame image in the image sequence to the pixel value of the complete frame, set the pixel value at the same pixel bit order of the next frame image adjacent to the complete frame image in the image sequence to the pixel value of the next frame, subtract the pixel value of the complete frame from the pixel value of the next frame to obtain the pixel difference in the image sequence;

[0051] Then store the difference of the pixel at the corresponding pixel position in the next frame image.

[0052] If the pixel in the first row is p0 （0，j） ; then the pixel in the first row of the nearest complete frame is p1 （0，j） ;

[0053] For the difference h （0，j） ;

[0054] h （0，0） = p0 （0，0） - p1 （0，0） ;

[0055] h （0，1） = p0 （0，1） - p1 （0，1） ;

[0056] ……;

[0057] h （0，j） = p0 （0，j） - p1 （0，j） ;

[0058] Convert the calculated h （0，j） to the pixel format with the number of data bits corresponding to the original image pixel value. For example, the p0 （0，0） pixel is 420, and its storage method is "110100100"; the p1 （0，0）） pixel is 422, and its storage method is "110100110"; for the calculated pixel difference of 2; store the difference 2 in the corresponding position of p1 （0，j） . The storage method is "000000010"; write "000000010" into the PSRAM.

[0059] In the raw data of the sensor, in the Bayer format, the difference between most pixels in the front and back frames is extremely small. For example, if the data of the same pixel in the previous frame is 450, the data of the same pixel in the next frame may be 448 or 452. Utilizing the above characteristics, first store a complete raw data frame. For the pixel data of subsequent frames, subtract the pixel values of the same pixels in the previous complete raw data frame, and only store the differences. The subsequent pixels store +2 or -2. Since the numerical values become smaller, the storage bits required change from the original complete 10-bit / 12-bit to 0-bit, 1-bit, or 2-bit, which can reduce a large amount of storage space, and the maximum space savings can reach 1 / 5 of the original.

[0060] Embodiment 2

[0061] Based on Embodiment 1, the storage according to the pixel differences in the image sequence in this embodiment includes:

[0062] Obtain the maximum pixel difference among the pixel differences in each group of pixels in each row, and determine the storage length occupied by the pixel differences of each pixel in this group in signed data format, that is, the number of bits occupied.

[0063] Convert the obtained pixel difference at each position into the pixel content of this data length and store it in the pixel bit order of this image sequence.

[0064] For each pixel h in the first group of the first row （0，0）， h （0，1），…… h （0，k） Difference:

[0065] h （0，0） = p0 （0，0） - p1 （0，0） ;

[0066] h （0，1） = p0 （0，1） - p1 （0，1） ;

[0067] ……;

[0068] h （0，k） = p0 （0，k） - p1 （0，k） ;

[0069] where hmax = max(h （0，0） , h （0，1） …h （0，k） ); According to the calculated maximum difference, determine the data length of hmax as L; and convert the pixel differences of other bits of h （0，0）， h （0，1）…… h （0，k） to this data length L;

[0070] For example: (h （0，0） , h（0，1） …h （0，k） ) = (0, 2, 4...4); Then the maximum difference, hmax corresponds to a data length of 3 bits, h （0，0）， h （0，1）…… h （0，k） All are converted to 3-bit data for storage. h （0，0） Converted to binary "000" for storage; h （0，1） Converted to binary "010" for storage; h （0，2） Converted to binary "100" for storage; Similarly, h （0，k） Converted to binary "100" for storage.

[0071] Example 3

[0072] Based on the above embodiments, forming different pixel bit orders in this embodiment further includes grouping a frame of image to obtain a grouped image sequence.

[0073] Storing the grouped image sequence includes:

[0074] Obtaining the pixel differences in each group of image sequences; and obtaining the maximum pixel difference from the pixel differences in each group, determining the storage length of the image pixels in each group of image sequences according to the maximum pixel difference; converting the pixel differences in the image sequence of this group into pixel content of this data length and storing it in the pixel bit order of this group.

[0075] For example, the pixels in the first row are p0 （0，j） ; Group the pixels in the first row, divide them into a groups, with k pixel data in each group, and a * k = j. The pixels in the first row of the nearest complete frame are p1 （0，j） ; Similarly, group the pixels in the first row of the nearest complete frame, divide them into a groups, with k pixel data in each group, and a * k = j;

[0076] For this, the differences in each group are the first group h （0，k） is (h （0，1）， h （0，2） ...h （0，k） ); The second group h （0，2k） is (h （0，k+1）， h （0，k+2） ...h （0，2k） ); The a-th group h （0，ak） is (h （0，ak-k+1）， h （0，ak-k+1） ...h （0，ak） );

[0077] Respectively determine h （0，k） h （0，2k） ...h （0，ak） group's maximum pixel difference hmax （0，k）, hmax （0，2k） …hmax （0，ak） ;

[0078] For example: (h （0，0） , h （0，1）， …, h （0，k） ) = (0, 2, 4……4); then the maximum difference, the data length corresponding to hmax is 3 bits, and h （0，0）， h （0，1）…… h （0，k） are all converted to 3-bit data for storage. h （0，0） is converted to binary "000" for storage; h （0，1） is converted to binary "010" for storage; h （0，2） is converted to binary "100" for storage; similarly, h （0，k） is converted to binary "100" for storage.

[0079] The above divides a row of pixels into a groups. For example, for 2 million pixels (row width is 1080), it can be divided into 20 groups, a = 20; each group has 54 pixels. After a row of pixels is exposed, gain amplified, and ADC converted, the pixel values of the corresponding row of the nearest complete frame in the storage space are subtracted to obtain the difference of each pixel; analyze the number of bits occupied by the maximum difference of the pixels in each group to obtain the space occupied by each pixel in this group; store the number of bits occupied by each group of pixels and the difference of each pixel in the corresponding space position, and the storage format is as Figure 3 shown in the schematic diagram of the compressed frame storage.

[0080] Example 4

[0081] Based on the above embodiment, the storage of the compressed frame in this embodiment further includes judging the total length of the compressed frame storage. When the total length of the compressed frame storage reaches the set storage length threshold, the next frame is stored as a complete frame, otherwise, the compressed frame storage continues. When the total length of the compressed frame storage is relatively large (for example, for a 2 million pixel sensor with a bit depth of 10 bits per pixel, the complete frame requires about 2.5 MB of storage space. When the total length of the compressed frame storage exceeds 1 MB and reaches the set storage length threshold, it indicates that the static state of the current picture has changed, and the next frame storage can be a complete frame storage.

[0082] After the master control end is awakened and obtains all the stored frame data, it restores frame by frame. For the compressed frame, the differences stored for each pixel are added to the corresponding pixel values of the nearest complete frame to losslessly restore the actual data.

[0083] Figure 3In [the above], for the stored pixel group of the compressed frame, the number of bits per pixel is fixed at 4 bits for representation. Its value indicates the number of bits occupied by each pixel in the group. If it is 0, it indicates that each pixel in the group does not need to occupy space. When it is other values, it indicates that the stored difference occupies space. Taking the largest occupied space of the pixel differences in each group to conform to binary as the standard, for example, if the maximum value is 2, it indicates that the occupied space is 2 bits.

[0084] When the main control unit sends a wake-up signal, the sensor stops pre-recording and starts to mark according to each stored frame. Based on the mark, it determines whether it is a complete frame storage or a compressed frame storage. For the storage of the compressed frame, the difference data is taken row by row and group by group, and added to the corresponding pixel values of the complete frame to restore to the original pixel data. For the complete frame data, the data of the complete frame is directly obtained.

[0085] Embodiment 5

[0086] Based on the above embodiments, this embodiment is a lossless video storage system under pre-recording. When the sensor pre-records the video, for the storage of the video, it includes:

[0087] Complete frame storage module:

[0088] During the video pre-recording process, each frame of the image constituting the video is arranged according to bit order and / or time sequence to form different image sequences; among them, the total number of frames of the image sequence is N frames; in the image sequence, the pixel contents of each frame of the image are arranged according to bit order and / or time sequence to form different pixel bit orders, and the first frame of the image obtained is stored as a complete frame according to the pixel content of the frame.

[0089] Compressed frame storage module:

[0090] In the image sequence, the pixel difference corresponding to the nearest complete frame is obtained according to the pixel content of each compressed frame, and the pixel difference of this frame is stored. When the storage space of the current compressed storage frame reaches the set threshold, the next frame is stored as a complete frame.

Claims

1. A method for storing pre-recorded lossless video. When the sensor pre-records video, after the exposure of each row of pixels is completed and the pixels of that row are gain-amplified, before the next row of pixels arrives, when the current frame is a complete frame, the data of the current row is stored in the SPRAM. When the current frame is a compressed frame, the data of the current row after processing the pixels of the current frame is stored in the SPRAM again; The storage of the video includes: Storage of complete frames: During the pre-recording of the video, each frame image that makes up the video is arranged according to bit order and / or time sequence to form different image sequences; among them, the total number of frames in the image sequence is N frames; in the image sequence, the pixel contents that make up each frame image are arranged according to bit order and / or time sequence to form different pixel bit orders, and the first frame image obtained is stored as a complete frame according to the pixel content of this frame image; Storage of compressed frames: In the image sequence, the pixel difference at the corresponding position of the nearest complete frame is obtained according to the pixel content of each compressed frame, and this frame of pixel difference is stored; when the storage space of the compressed storage frame reaches the set threshold value, the next frame is stored as a complete frame; the storage of compressed frames includes: Set the pixel value at the first pixel position of the complete frame image in the image sequence to the pixel value of this complete frame, set the pixel value of the same pixel bit order of the next frame image adjacent to the complete frame image in this image sequence to the pixel value of the next frame, subtract the pixel value of this complete frame from the pixel value of the next frame to obtain the pixel difference located in this image sequence; then store the difference of this pixel at the corresponding pixel position in the next frame image; the storage according to the pixel difference in the image sequence includes: Obtain the largest pixel difference in the pixel differences and determine the data length of the largest pixel difference; Convert the pixel difference at each obtained position into the pixel content of this data length and store it in the pixel bit order of this image sequence.

2. The lossless video storage method of pre-recording according to claim 1, wherein, Forming different pixel bit orders also includes grouping a frame of image to obtain a grouped image sequence.

3. A method for storing pre-recorded lossless videos according to claim 2, characterized in that, The storage of the grouped image sequence includes: Obtain the pixel differences in each group of image sequences; And obtain the largest pixel difference from the pixel differences in each group, and determine the storage length of the image pixels in each group of image sequences according to this largest pixel difference; Convert the pixel differences in the image sequence of this group into the pixel content of this data length and store it in the pixel bit order of this group.

4. A method for storing pre-recorded lossless videos according to claim 1, characterized in that, The storage of compressed frames also includes judging the total length of the compressed frame storage. When the total length of the compressed frame storage reaches the set storage length threshold, the next frame is stored as a complete frame, otherwise, the storage of compressed frames continues.

5. A pre-recorded lossless video storage system, characterized in that, When the sensor pre-records the video, after the exposure of each row of pixels is completed and the pixels of this row are gain-amplified, before the next row of pixels arrives, when the current frame is a complete frame, the current row data is stored in the SPRAM, and when the current frame is a compressed frame, the current row data after processing the current frame pixels is stored in the SPRAM again; The storage of the video includes: Complete frame storage module: During the pre-recording of the video, each frame image that makes up the video is arranged according to bit order and / or time sequence to form different image sequences; among them, the total number of frames in the image sequence is N frames; in the image sequence, the pixel contents that make up each frame image are arranged according to bit order and / or time sequence to form different pixel bit orders, and the first frame image obtained is stored as a complete frame according to the pixel content of this frame image; Compressed frame storage module: In the image sequence, the pixel difference from the corresponding position of the nearest complete frame is obtained according to the content per pixel of the compressed frame, and the pixel difference of this frame is stored; when the storage space of the current compressed storage frame reaches the set threshold, the next frame is stored as a complete frame; the storage of the compressed frame includes: Set the pixel value at the first pixel position of the complete frame image in the image sequence to the pixel value of the complete frame, set the pixel value at the same pixel order of the next frame adjacent to the complete frame image in the image sequence to the pixel value of the next frame, subtract the pixel value of the complete frame from the pixel value of the next frame to obtain the pixel difference in the image sequence; then store the difference of this pixel at the corresponding pixel position in the next frame image; storing according to the pixel difference in the image sequence includes: Obtain the largest pixel difference among the pixel differences and determine the data length of the largest pixel difference; Convert the pixel difference at each obtained position into the pixel content of this data length and store it in the pixel order of the image sequence.

Images