Video data processing method and device, equipment and storage medium

Abstract

The invention provides a video data processing method and device, equipment and a storage medium. The method is applied to video coding technology. The method comprises the following steps: when the video equipment is awakened, acquiring encoder state information and a reference frame stored in the last awakening, reloading an encoder according to the encoder state information, if the encoding type is determined to be inter-frame predictive encoding based on the encoder state information, performing inter-frame encoding on video data acquired by the awakening according to the reference frame, and if the encoding type is determined to be inter-frame predictive encoding, performing inter-frame encoding on the video data acquired by the awakening according to the reference frame. And generating a prediction frame, setting the YUV data corresponding to the prediction frame as a new reference frame, setting the encoder state information of the encoder after the encoding as new encoder state information, storing the new reference frame and the new encoder state information in a nonvolatile memory, and performing power-off dormancy. Through the method, the compression ratio is improved, the energy consumption is reduced, the coding efficiency is also improved in a low-power-consumption full-time video recording scene, and the video quality is improved through flexible reference frame management.

Description

Technical Field

[0001] The present application relates to the field of video coding technology, and in particular to a method, device, equipment and storage medium for processing video data. Background Art

[0002] Full-time recording in low-power mode is an advanced surveillance technology that can achieve all-weather recording while significantly reducing energy consumption. Low-power technology allows surveillance equipment to be in a dormant or low-power state most of the time, and only wake up and start recording when a specific event is detected.

[0003] When the video recording device wakes up, the video recording device collects data, then performs intra-frame encoding on the collected data and saves the final video code stream data.

[0004] However, the video code stream data bit rate obtained by the above method is relatively high, which not only increases the storage and transmission burden, but also increases power consumption. Summary of the invention

[0005] The present application provides a method, device, equipment and storage medium for processing video data, so as to solve the problem that the video code stream data bit rate obtained by encoding the video data in the prior art is relatively large, which not only increases the storage and transmission burden, but also increases the power consumption.

[0006] In a first aspect, the present application provides a method for processing video data, which is applied to a video recording device, and the method comprises:

[0007] When the video recording device wakes up, the encoder state information and reference frame stored in the last wake-up are obtained;

[0008] Reloading the encoder according to the encoder status information;

[0009] Based on the encoder state information, if it is determined that the encoding type of this wake-up is inter-frame prediction encoding, inter-frame encoding is performed on the video data collected by this wake-up according to the reference frame to generate a prediction frame;

[0010] The YUV data corresponding to the predicted frame is set as a new reference frame, and the encoder state information of the encoder after the current encoding is set as the new encoder state information;

[0011] The new reference frame and the new encoder state information are stored in a non-volatile memory, and the system is powered off and put into sleep mode.

[0012] In combination with the first aspect, in some embodiments, the method further includes:

[0013] Based on the encoder state information, if it is determined that the encoding type of this wake-up is intra-frame prediction encoding, intra-frame encoding is performed on the video data to generate a key frame;

[0014] The YUV data corresponding to the key frame is set as the new reference frame.

[0015] In combination with the first aspect, in some embodiments, the method further includes:

[0016] After the encoder is reloaded, the number of inter-frame prediction encodings between the last intra-frame prediction encoding and the current wake-up is obtained;

[0017] If the number is less than the preset picture group size, it is determined that the encoding type of this wake-up is inter-frame prediction encoding.

[0018] In combination with the first aspect, in some embodiments, the method further includes:

[0019] If the number is equal to the preset picture group size, it is determined that the encoding type of this wake-up is intra-frame prediction encoding.

[0020] In combination with the first aspect, in some embodiments, based on the encoder state information, if it is determined that the encoding type of this wake-up is inter-frame prediction encoding, inter-frame encoding is performed on the video data collected by this wake-up according to the reference frame to generate a prediction frame, including:

[0021] Dividing the video data into a plurality of pixel blocks;

[0022] For each pixel block, based on a preset search range, searching in the reference frame to obtain a matching block corresponding to the pixel block;

[0023] Based on the pixel block, determining a position offset of a matching block corresponding to the pixel block relative to the pixel block;

[0024] Reconstructing a prediction block of the pixel block in the reference frame according to the position offset, and determining a residual block between the prediction block and the pixel block;

[0025] The position offset and residual block corresponding to each pixel block are encoded and assembled to obtain the predicted frame.

[0026] In combination with the first aspect, in some embodiments, if it is determined that the encoding type of this wake-up is intra-frame prediction encoding based on the encoder state information, intra-frame encoding is performed on the video data to generate a key frame, including:

[0027] Dividing the video data into a plurality of pixel blocks;

[0028] For each pixel block, determining a predicted value of the pixel block according to pixel values ​​of adjacent pixel blocks of the pixel block;

[0029] Determining a residual block according to the pixel value of the pixel block and the prediction value;

[0030] The residual block of each pixel block is transformed, quantized and encoded, and the encoded information is assembled to obtain the key frame.

[0031] In combination with the first aspect, in some embodiments, the method further includes:

[0032] When a preset wake-up period is reached, the video recording device wakes up;

[0033] or;

[0034] When a preset target moving object and / or target sound is detected, the video recording device wakes up.

[0035] In a second aspect, the present application provides a video data processing device, comprising:

[0036] A first acquisition module, used for acquiring, when the video recording device wakes up, encoder state information and reference frames stored in the last wake-up;

[0037] A reload module, used for reloading the encoder according to the encoder status information;

[0038] A first generating module is used for, based on the encoder state information, if it is determined that the encoding type of this wake-up is inter-frame prediction encoding, then inter-frame encoding is performed on the video data collected by this wake-up according to the reference frame to generate a prediction frame;

[0039] A first updating module, used to set the YUV data corresponding to the predicted frame as a new reference frame, and set the encoder state information of the encoder after the current encoding as the new encoder state information;

[0040] The storage module is used to store the new reference frame and the new encoder state information in a non-volatile memory and power off and enter sleep mode.

[0041] In conjunction with the second aspect, in some embodiments, the apparatus further includes:

[0042] A second generating module is used for performing intra-frame encoding on the video data to generate a key frame if it is determined that the encoding type of this wake-up is intra-frame prediction encoding based on the encoder state information;

[0043] The second updating module is used to set the YUV data corresponding to the key frame as the new reference frame.

[0044] In conjunction with the second aspect, in some embodiments, the apparatus further includes:

[0045] The second acquisition module is used to acquire the number of inter-frame prediction encodings between the last intra-frame prediction encoding and the current wake-up after the encoder completes the reload;

[0046] The first determination module is used to determine that the encoding type of this wake-up is inter-frame prediction encoding if the number is less than a preset picture group size.

[0047] In conjunction with the second aspect, in some embodiments, the apparatus further includes:

[0048] The second determination module is used to determine that the encoding type of this wake-up is intra-frame prediction encoding if the number is equal to a preset picture group size.

[0049] In conjunction with the second aspect, in some embodiments, the first generating module includes:

[0050] A division unit, used for dividing the video data into a plurality of pixel blocks;

[0051] A searching unit, configured to search for each pixel block in the reference frame based on a preset search range to obtain a matching block corresponding to the pixel block;

[0052] a determining unit, configured to determine, based on the pixel block, a position offset of a matching block corresponding to the pixel block relative to the pixel block;

[0053] a reconstruction unit, configured to reconstruct a prediction block of the pixel block in the reference frame according to the position offset, and determine a residual block between the prediction block and the pixel block;

[0054] The assembling unit is used to encode and assemble the position offset and residual block corresponding to each pixel block to obtain the predicted frame.

[0055] In conjunction with the second aspect, in some embodiments, the second generating module includes:

[0056] A division unit, used for dividing the video data into a plurality of pixel blocks;

[0057] A first determining unit, configured to determine, for each pixel block, a predicted value of the pixel block according to pixel values ​​of adjacent pixel blocks of the pixel block;

[0058] A second determining unit, configured to determine a residual block according to a pixel value of the pixel block and the prediction value;

[0059] The assembling unit is used to transform, quantize and encode the residual block of each pixel block, and assemble the encoded information to obtain the key frame.

[0060] In conjunction with the second aspect, in some embodiments, the apparatus further includes:

[0061] A first wake-up module, used to wake up the video recording device when a preset wake-up period is reached;

[0062] or;

[0063] The second wake-up module is used to wake up the video recording device when a preset target moving object and / or target sound is detected.

[0064] In a third aspect, the present application provides a video recording device, comprising: a processor, and a memory, a communication interface, and a camera connected to the processor;

[0065] The memory stores computer-executable instructions;

[0066] The processor executes the computer-executable instructions stored in the memory to implement the method for processing video data described in any one of the first aspects.

[0067] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method for processing video data described in any of the aforementioned aspects.

[0068] In a fifth aspect, the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the method for processing video data described in any of the above aspects.

[0069] The processing method, device, equipment and storage medium of video data provided by the present application obtain the encoder status information and reference frame stored in the last wake-up when the video recording device wakes up, and reloads the encoder according to the encoder status information. Based on the encoder status information, if the encoding type is determined to be inter-frame prediction encoding, inter-frame encoding is performed on the video data collected during this wake-up according to the reference frame to generate a prediction frame, and the YUV data corresponding to the prediction frame is set as a new reference frame, and the encoder status information of the encoder after this encoding is set as the new encoder status information, and the new reference frame and the new encoder status information are stored in a non-volatile memory, and the device is powered off and put into sleep mode. Through the above method, the compression rate is improved, the energy consumption is reduced, and the encoding efficiency is improved in the low-power full-time video recording scenario, and the flexible reference frame management improves the video quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0071] Figure 1 An application scenario diagram of the method for processing video data provided in an embodiment of the present application;

[0072] Figure 2 A flowchart of a first embodiment of a method for processing video data provided in an embodiment of the present application;

[0073] Figure 3 A flowchart of a second embodiment of a method for processing video data provided in an embodiment of the present application;

[0074] Figure 4 A flowchart of a third embodiment of a method for processing video data provided in an embodiment of the present application;

[0075] Figure 5 A flowchart of a fourth embodiment of a method for processing video data provided in an embodiment of the present application;

[0076] Figure 6 A flowchart of a fifth embodiment of a method for processing video data provided in an embodiment of the present application;

[0077] Figure 7 A flowchart of a specific example of a method for processing video data provided in an embodiment of the present application;

[0078] Figure 8 A schematic diagram of the structure of a first embodiment of a device for processing video data provided in an embodiment of the present application;

[0079] Fig. 9 A schematic diagram of the structure of a second embodiment of a device for processing video data provided in an embodiment of the present application;

[0080] Fig.10 A schematic diagram of the structure of a third embodiment of a device for processing video data provided in an embodiment of the present application;

[0081] Fig.11 A schematic diagram of the structure of a fourth embodiment of a device for processing video data provided in an embodiment of the present application;

[0082] Fig.12 A schematic diagram of the structure of a fifth embodiment of a device for processing video data provided in an embodiment of the present application;

[0083] Fig.13 A schematic diagram of the structure of a sixth embodiment of a device for processing video data provided in an embodiment of the present application;

[0084] Fig.14 A schematic diagram of the structure of a video recording device provided in an embodiment of the present application.

[0085] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0086] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0087] Full-time recording in low-power mode is an advanced surveillance technology that can achieve all-weather recording while significantly reducing energy consumption. Low-power technology allows surveillance equipment to be in a dormant or low-power state most of the time, and only wake up and start the recording function when a specific event is detected. When the recording device wakes up, the recording device collects data, then encodes the collected data intra-frame and saves the final video stream data. This method is relatively simple and has no special dependence on hardware, but because all frames are intra-frame predictive encoding, the resulting video stream data has a large bit rate, which not only increases the storage and transmission burden, but also increases power consumption.

[0088] In view of the above problems, the present application provides a method, device, equipment and storage medium for processing video data, thereby reducing the bit rate of video code stream data, ensuring low power consumption, and reducing memory consumption. Specifically, in the prior art, after the device wakes up, the collected picture is intra-frame predictive encoded to generate key frames. This method is simple and has no special dependence on hardware. However, since all frames are intra-frame predictive encoded, the final code stream data will be large. If after the device wakes up, only data is collected without encoding, and the collected original data is directly saved, when the accumulated saved data reaches a certain amount after multiple wake-ups, all the saved data are encoded, but this method saves a large amount of original data and consumes a large amount of storage space. Considering these problems, the inventors have studied whether it is possible to preset a picture group and reduce the bit rate of the video code stream by hybrid encoding, thereby ensuring low power consumption, and in each encoding process, the YUV data of a frame of data encoded this time is used as the reference frame for the next encoding, and after each encoding is completed, the reference frame is deleted and replaced, thereby reducing the consumption of storage resources. Based on this, the technical solution of the present application is proposed.

[0089] Figure 1 This is an application scenario diagram of the method for processing video data provided in the embodiment of the present application, such as Figure 1 As shown, the scenario includes at least a recording device and a remote device, wherein the remote device can be an electronic device such as a server, cloud, smart phone, laptop, etc. The recording device is in a low-power sleep state for a long time. The user can set the wake-up method. When the wake-up conditions are met, the recording device automatically wakes up to record, collects recording data, and then sends the video stream to the remote device after hybrid encoding when the video stream reaches a certain amount or is sent at a fixed time. The user can view the recording information on the remote device.

[0090] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0091] Figure 2 The flowchart of the first embodiment of the method for processing video data provided in the present application is as follows: Figure 2 As shown, the method can be used for a video recording device, and the method specifically includes:

[0092] S201: When the video recording device wakes up, the encoder state information and reference frame stored in the last wake-up are obtained.

[0093] S202: Reload the encoder according to the encoder status information.

[0094] Low-power devices cannot be powered on all the time because of battery life considerations. Therefore, the implementation method of full-time recording of low-power devices is generally to power on in a certain period to encode a frame of data, then power off, and then power on again in the next period to encode a frame. However, since the status and reference frame information saved in the encoder will be lost when the power is off, the encoder needs to be reset every time it is powered on. In order to reduce the bit rate of the encoded bit stream data, the status information and reference frame after each encoding can be stored in a non-volatile memory. When the recording device wakes up, the encoder status information and reference frame stored in the last wake-up are obtained, and then the encoder is reloaded according to the encoder status information.

[0095] S203: Based on the encoder state information, if it is determined that the encoding type of this wake-up is inter-frame prediction encoding, inter-frame encoding is performed on the video data collected by this wake-up according to the reference frame to generate a prediction frame.

[0096] In this step, in order to reduce the bit rate of the encoded video stream, a hybrid encoding method can be used to ensure low power consumption. The video recording device is in a low-power standby state most of the time and wakes up under specific conditions. After the video recording device wakes up and completes the encoder reload, the video recording device collects video data and then determines the encoding type of this encoding. If it is determined that the encoding type of this encoding is inter-frame prediction encoding, the reference frame stored in the last wake-up is obtained, and then the video data is inter-frame encoded to obtain a predicted frame.

[0097] Specifically, the video data is divided into a plurality of pixel blocks, and for each pixel block, based on a preset search range, a search is performed in a reference frame to obtain a matching block corresponding to the pixel block, and based on the pixel block, a position offset of the matching block corresponding to the pixel block relative to the pixel block is determined, and according to the position offset, a prediction block of the pixel block is reconstructed in the reference frame, and a residual block between the prediction block and the pixel block is determined, and the position offset and the residual block corresponding to each pixel block are encoded and assembled to obtain a prediction frame.

[0098] Optionally, when the video recording device is in sleep mode, it will be awakened if any of the following conditions are met:

[0099] 1. When the preset wake-up cycle is reached, the recording device wakes up.

[0100] Exemplarily, the recording device automatically wakes up to collect data according to a preset schedule. The user can set the recording device to automatically wake up and start collecting data at a specific time (such as early morning or daytime every day), and automatically enter a sleep state after the preset time, or set it to automatically wake up and collect data every 2 hours, and automatically enter a sleep state after 10 minutes.

[0101] 2. When a preset target moving object and / or target sound is detected, the recording device wakes up.

[0102] For example, the video recording device is internally integrated with a motion detection sensor or algorithm, which can monitor the motion changes in the monitoring area in real time. When the motion in the preset area reaches a certain level (such as the pixel change rate exceeds a threshold), the video recording device will automatically wake up from the low-power standby state and start collecting data.

[0103] You can also rely on sound sensors to detect changes in ambient sound. When the sound sensor detects sounds above a preset decibel, it triggers the video recording device to wake up and start collecting data.

[0104] In one possible implementation, the user can also control the recording device to wake up through remote commands or applications. The recording device is connected to the Internet or a local area network, and the user can send a wake-up command through a mobile phone application, a web page, or other remote control terminal. After receiving the command, the recording device automatically wakes up and starts collecting data.

[0105] It can also be automatically awakened according to changes in environmental parameters (such as light, temperature, etc.). The recording device is integrated with corresponding environmental sensors. When it detects that the environmental parameters reach the preset threshold (such as the light suddenly dims, the temperature rises sharply, etc.), the recording device is triggered to wake up and start collecting data.

[0106] Or it can be automatically awakened based on the occurrence of specific events. This can include smoke detector triggering (fire), intruder detector triggering (security system), etc. When these specific events occur, the recording device will automatically wake up and start collecting data.

[0107] S204: Set the YUV data corresponding to the predicted frame as a new reference frame, and set the encoder state information of the encoder after the current encoding as the new encoder state information.

[0108] S205: Store the new reference frame and the new encoder state information in the non-volatile memory, and power off and enter sleep mode.

[0109] In this step, after completing the inter-frame prediction coding of the current recording data, in order to save storage space and thus improve coding efficiency, the reference frame stored in the last wake-up can be deleted, so that the YUV data of the original data corresponding to the currently encoded recording data can be set as a new reference frame for preservation, and then used for the next wake-up coding. In order for the next recording device to be powered on continuously with the last power-on, the encoder status information after this encoding is stored in the non-volatile memory as the new status information and the new reference frame, and then powered off and put into sleep mode until the next wake-up.

[0110] Specifically, after the encoding is completed, the video data collected this time is copied to the memory every day, and then converted into YUV format data and saved in the memory as a new reference frame, and the original reference frame is deleted. The memory is a memory that does not lose power, such as a disk.

[0111] The method for processing video data provided in this embodiment obtains the encoder status information and reference frame stored in the last wake-up when the video recording device wakes up, and reloads the encoder according to the encoder status information. Based on the encoder status information, if the encoding type is determined to be inter-frame prediction encoding, inter-frame encoding is performed on the video data collected during this wake-up according to the reference frame to generate a prediction frame, and the YUV data corresponding to the prediction frame is set as a new reference frame, and the encoder status information of the encoder after this encoding is set as the new encoder status information, and the new reference frame and the new encoder status information are stored in a non-volatile memory, and the device is powered off and put into sleep mode. Through the above method, the compression rate is improved, the energy consumption is reduced, and the encoding efficiency is improved in the low-power full-time video recording scenario, and the flexible reference frame management improves the video quality.

[0112] Figure 3 A flowchart of a second embodiment of a method for processing video data provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, based on the above embodiment, the method further includes:

[0113] S301: Based on the encoder state information, if it is determined that the encoding type of this wakeup is intra-frame prediction encoding, intra-frame encoding is performed on the video data to generate a key frame.

[0114] In this step, in order to reduce the bit rate of the encoded video stream, a hybrid encoding method can be used to ensure low power consumption. The video recording device is in a low-power standby state most of the time and wakes up under specific conditions. When the video recording device wakes up, the video recording device collects video data and then determines the encoding type of this encoding. If the encoding type is determined to be intra-frame prediction encoding, the video data is directly intra-encoded to generate key frames.

[0115] Specifically, the video data is divided into multiple pixel blocks. For each pixel block, the prediction value of the pixel block is determined according to the pixel values ​​of the adjacent pixel blocks of the pixel block. The residual block is determined according to the pixel value and the prediction value of the pixel block. The residual block of each pixel block is transformed, quantized and encoded, and the encoded information is assembled to obtain a key frame.

[0116] S302: Set the YUV data corresponding to the key frame as a new reference frame.

[0117] In this step, after the current encoding is completed, in order to save storage space and improve encoding efficiency, the reference frame stored in the last wake-up is deleted, and the YUV data corresponding to the key frame obtained in this encoding is set as a new reference frame.

[0118] It should be noted that if the recording device is awakened for the first time, there is no reference frame stored in the memory, and there is no need to delete the reference frame. The YUV data corresponding to the key frame obtained by this encoding is directly set as the reference frame.

[0119] In the method for processing video data provided in this embodiment, if the encoding type is determined to be intra-frame prediction encoding, intra-frame encoding is performed on the video data to generate a key frame, the reference frame is deleted, and the YUV data corresponding to the key frame is set as a new reference frame. Through the above method, the compression rate is improved, the transmission bandwidth and storage space are reduced, and the decoding efficiency is improved, the error recovery capability, flexibility and adaptability are enhanced in the full-time video recording scenario in the low power consumption mode.

[0120] Figure 4 The flowchart of the third embodiment of the method for processing video data provided in the embodiment of the present application is as follows: Figure 4As shown, based on the above embodiment, the method further includes:

[0121] S401: After the encoder completes reloading, the number of inter-frame prediction encodings between the last intra-frame prediction encoding and the current wake-up is obtained.

[0122] S402: If the number is less than the preset picture group size, determine that the encoding type for this wakeup is inter-frame prediction encoding.

[0123] S403: If the number is equal to the preset picture group size, it is determined that the encoding type of this wake-up is intra-frame prediction encoding.

[0124] After the video recording device wakes up and completes the encoder reload, the video recording device collects the video data, and then determines the encoding type of this encoding. It needs to obtain the number of inter-frame prediction encodings between the last intra-frame prediction encoding and this wake-up, and then compare it with the preset picture group size. The picture group is a group of continuous frames. Usually, a picture group starts with a key frame, followed by a series of non-key frames, that is, prediction frames, until the next key frame.

[0125] If the number is less than the preset GOP size, interframe prediction coding can continue to be used to reduce the frequency of key frame generation. The encoding type for this wakeup is determined to be interframe prediction coding. If the number is equal to the GOP size, it means that a sufficient number of non-key frames have been encoded since the last key frame. By inserting a new key frame, the image quality in the video stream can be updated in time and can better adapt to the encoding of subsequent frames. Inserting a key frame when the GOP size is reached can improve the error recovery capability of the video stream. The encoding type for this wakeup is determined to be intraframe prediction coding.

[0126] The method for processing video data provided in this embodiment obtains the number of inter-frame prediction coding between the last intra-frame prediction coding and the current wake-up when the video recording device wakes up. If the number is less than the preset picture group size, the coding type of this wake-up is determined to be inter-frame prediction coding. If the number is equal to the picture group size, the coding type of this wake-up is determined to be intra-frame prediction coding. By comparing the number of inter-frame prediction coding with the preset picture group size, the coding strategy is automatically adjusted, thereby meeting the requirement of the video recording device for low energy consumption. By using the preset picture group size as a threshold, the generation of key frames can be controlled while maintaining a certain compression efficiency. It helps to find a balance between compression efficiency and energy consumption, without excessive energy consumption or excessive sacrifice of video quality.

[0127] Figure 5 The flowchart of the fourth embodiment of the method for processing video data provided in the embodiment of the present application is as follows: Figure 5As shown, based on the above embodiments, step S203 specifically includes:

[0128] S501: Divide the video data into a plurality of pixel blocks.

[0129] In this step, the video data (i.e., video frames) are divided into a plurality of pixel blocks (also commonly referred to as macroblocks, coding units, or prediction units). The size of these blocks can be fixed or adaptive, depending on the coding standard and the characteristics of the video content. For example, 16x16 pixels, 8x8 pixels, or smaller 4x4 pixel blocks. The embodiments of the present application are not specifically limited.

[0130] S502: For each pixel block, based on a preset search range, search in the reference frame to obtain a matching block corresponding to the pixel block.

[0131] In this step, a search range is set for each pixel block, which is usually near the corresponding position of the previous frame (or previous frames) of the current frame. For each pixel block, within the preset search range, a certain matching criterion (such as minimum mean square error MSE, minimum mean absolute error MAD, etc.) is used to find the matching block that is most similar to the current pixel block in the reference frame.

[0132] Exemplarily, a search algorithm is used to find the best matching block within the search range. The search algorithm may be a full search (FS), a fast search (such as a diamond search, a hexagonal search, etc.) or other more efficient algorithms. This embodiment of the application is not specifically limited.

[0133] S503: Based on the pixel block, determine the position offset of the matching block corresponding to the pixel block relative to the pixel block.

[0134] In this step, after the matching block corresponding to each pixel block is obtained, the position offset of the matching block relative to the current pixel block is calculated.

[0135] For example, let the coordinate position of the current pixel block in the reference frame be (x 1 ,y 1 ), it should be noted that the coordinates here can be offsets relative to a reference point, and the coordinate position of the best matching block in the reference frame is (x 2 ,y 2 ).

[0136] Then, the horizontal and vertical components of the position offset can be calculated by the following formulas respectively:

[0137] MVx=x 2 -x 1

[0138] MVy=y 2 -y 1

[0139] Wherein, MVx represents the horizontal component of the position offset, and MVy represents the vertical component of the position offset.

[0140] S504: reconstructing a prediction block of the pixel block in the reference frame according to the position offset, and determining a residual block between the prediction block and the pixel block.

[0141] In this step, after the position offset corresponding to each pixel block is obtained, the prediction block of each pixel block is reconstructed in the reference frame according to the position offset of each pixel block, and then the residual block is calculated.

[0142] Exemplarily, for each pixel block, if the position offset of the pixel block is not an integer, that is, there is sub-pixel precision, then interpolation is required in the reference frame to generate pixel values ​​at the sub-pixel position, which can be achieved through an interpolation algorithm (such as bilinear interpolation, bicubic interpolation, etc.). If the position offset of the pixel block is an integer and there is no sub-pixel precision requirement, the pixel block at the corresponding position can be directly copied from the reference frame as a prediction block. If there is sub-pixel precision, after the pixel values ​​at the sub-pixel position are interpolated, the prediction block is generated based on these values ​​or may also include pixel values ​​at integer positions.

[0143] Optionally, in order to further improve the quality of the predicted block, it can also be post-processed, for example, by denoising, sharpening, color correction and other operations.

[0144] Finally, the pixel difference between the current pixel block and its predicted block is calculated to obtain a residual block, which also contains information about the unpredicted part of the current pixel block.

[0145] S505: Encode and assemble the position offset and residual block corresponding to each pixel block to obtain a predicted frame.

[0146] In this step, the position offset and residual block of each pixel block are encoded. The encoding process may include transformation (such as DCT transformation), quantization, entropy coding, etc. to further reduce the amount of data. The encoded position offset and residual block information are then assembled to form a predicted frame.

[0147] The method for processing video data provided in this embodiment divides the video data into a plurality of pixel blocks, searches for each pixel block in a reference frame based on a preset search range, obtains a matching block corresponding to the pixel block, determines the position offset of the matching block corresponding to the pixel block relative to the pixel block based on the pixel block, reconstructs the prediction block of the pixel block in the reference frame according to the position offset, determines the residual block between the prediction block and the pixel block, encodes and assembles the position offset and residual block corresponding to each pixel block, and obtains a prediction frame. Through the above method, time redundancy is removed, the amount of data is reduced, and image quality is guaranteed while having a high compression rate.

[0148] Figure 6 The flowchart of the fifth embodiment of the method for processing video data provided in the present application is as follows: Figure 6 As shown, based on the above embodiments, step S301 specifically includes:

[0149] S601: Divide the video data into a plurality of pixel blocks.

[0150] The specific implementation method of this step is the same as that of step S501 in the above embodiment, and will not be repeated here.

[0151] S602: For each pixel block, determine a prediction value of the pixel block according to pixel values ​​of adjacent pixel blocks of the pixel block.

[0152] In this step, for each pixel block, the information of adjacent pixel blocks is used to predict the value of the current pixel block, so as to reduce the amount of data required for encoding.

[0153] Specifically, according to the coding standard, an appropriate prediction mode is selected. For example, in H.264 / AVC, a luminance block can have multiple 4x4 and 16x16 prediction modes. The pixel values ​​of the adjacent pixel blocks (usually the left and upper pixel blocks) are used to calculate the prediction value of the current pixel block according to the selected prediction mode.

[0154] S603: Determine a residual block according to the pixel values ​​and the prediction values ​​of the pixel block.

[0155] For each pixel block, the difference between its actual value and the predicted value is calculated to obtain a residual block.

[0156] S604: transform, quantize and encode the residual block of each pixel block, assemble the encoded information to obtain a key frame.

[0157] In this step, the data volume of the residual block is further reduced through transformation and quantization, and is encoded for transmission and storage.

[0158] Specifically, discrete cosine transform (DCT) is used to transform the residual block from the spatial domain to the frequency domain. This helps to remove the spatial correlation in the residual block. The transformed coefficients are quantized according to a certain quantization step size to reduce the accuracy and dynamic range of the data. The larger the quantization step size, the higher the compression ratio. The quantized coefficients are encoded (such as using entropy coding methods such as Huffman coding and arithmetic coding) to generate the final encoded data, and then the encoded information corresponding to each pixel block is assembled to obtain a key frame.

[0159] The processing method of video data provided by this embodiment divides the video data into multiple pixel blocks, and for each pixel block, determines the predicted value of the pixel block according to the pixel values ​​of the adjacent pixel blocks of the pixel block, determines the residual block according to the pixel value and the predicted value of the pixel block, transforms, quantizes and encodes the residual block of each pixel block, and assembles the encoded information to obtain a key frame. Through the above method, intra-frame prediction encoding is performed based on the pixel values ​​of adjacent pixel blocks, thereby removing spatial redundancy, improving compression efficiency, improving image quality, and improving flexibility and facilitating decoding and error recovery.

[0160] Figure 7 A flowchart of a specific example of a method for processing video data provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, the method includes:

[0161] S701: The video recording device wakes up for the first time and performs intra-frame prediction coding.

[0162] The video recording device performs intra-frame prediction coding when it is first awakened. The specific implementation process is the same as the intra-frame prediction coding process in the aforementioned embodiment, which will not be repeated here.

[0163] S702: Save original data.

[0164] After the intra-frame prediction coding is completed, the YUV data of the original data of the key frame obtained by this coding is saved as a reference frame.

[0165] S703: Sleep.

[0166] S704: Wake up.

[0167] After completing one encoding, the video recording device enters a dormant state until the wake-up condition is met, and then collects video data after wake-up. The wake-up condition is the same as in the above embodiment, and will not be repeated here.

[0168] S705: Reload the encoder to determine whether intra-frame prediction coding is currently required.

[0169] Since this wake-up is not the first wake-up, the encoder state information and reference frame stored in the non-volatile memory after the last encoding need to be obtained first, and then whether intra-frame prediction encoding is required at present is determined according to the picture group size of the pre-equipped device. The specific determination process is the same as step S401 to step S403 in the above embodiment, and will not be repeated here.

[0170] S706: If intra-frame prediction coding is not currently required, the previously saved original data is used as a reference frame.

[0171] If it is determined that intra-frame prediction coding is not required, inter-frame prediction coding is performed. Before coding, a reference frame needs to be obtained, and the reference frame saved when first awakening is retrieved in advance.

[0172] S707: Inter-frame prediction coding.

[0173] The inter-frame prediction coding is performed according to the reference frame. The specific implementation process is the same as the inter-frame prediction coding process in the above embodiment, and will not be repeated here.

[0174] S708: If intra-frame prediction coding is currently required, intra-frame prediction coding is performed.

[0175] If it is determined that intra-frame prediction coding is currently required, intra-frame prediction coding is performed, and the specific coding process is not described in detail here.

[0176] S709: Delete the original data saved previously

[0177] S710: Save the original data of the current frame.

[0178] After the current encoding is completed, the last saved reference frame is deleted, the YUV data of the original data of the current frame is saved as a new reference frame, and then the system goes into sleep until the next wake-up.

[0179] The processing method of video data provided in this embodiment sets the size of the picture group and then determines the encoding type after each wake-up of the video recording device. After each encoding, the YUV data of the original data is saved as a reference frame, and the reference frame saved last time is deleted. The hybrid encoding method can not only reduce the bit rate of the video code stream, but also improve the compression rate, reduce the transmission bandwidth and storage space while ensuring the video quality.

[0180] Figure 8 The structure diagram of the first embodiment of the apparatus for processing video data provided in the present application is as follows: Figure 8 As shown, the video data processing device 800 includes:

[0181] The first acquisition module 801 is used to acquire the encoder state information and reference frame stored in the last wake-up when the video recording device wakes up.

[0182] The reload module 802 is used to reload the encoder according to the encoder status information.

[0183] The first generating module 803 is used to, based on the encoder state information, if it is determined that the encoding type is inter-frame prediction encoding, perform inter-frame encoding on the video data collected in this wake-up according to the reference frame stored in the previous wake-up to generate a prediction frame.

[0184] The first updating module 804 is used to set the YUV data corresponding to the predicted frame as a new reference frame, and set the encoder state information of the encoder after the current encoding as the new encoder state information.

[0185] The storage module 805 is used to store the new reference frame and the new encoder state information in the non-volatile memory and power off and enter sleep mode.

[0186] Fig. 9 The structural diagram of the second embodiment of the apparatus for processing video data provided in the embodiment of the present application is as follows: Fig. 9 As shown, based on the above embodiment, the video data processing device 800 further includes:

[0187] The second generating module 901 is used for performing intra-frame encoding on the video data to generate key frames if the encoding type is determined to be intra-frame prediction encoding based on the encoder state information.

[0188] The second updating module 902 is used to set the YUV data corresponding to the key frame as a new reference frame.

[0189] Fig.10 The schematic diagram of the structure of the third embodiment of the video data processing device provided in the embodiment of the present application is as follows: Fig.10 As shown, based on the above embodiment, the video data processing device 800 further includes:

[0190] The second acquisition module 1001 is used to acquire the number of inter-frame prediction encodings from the last intra-frame prediction encoding to the current wake-up after the encoder completes reloading.

[0191] The first determining module 1002 is configured to determine that the encoding type for this wakeup is inter-frame prediction encoding if the number is less than a preset picture group size.

[0192] Optionally, the video data processing device 800 further includes:

[0193] The second determination module 1003 is configured to determine that the encoding type for this wakeup is intra-frame prediction encoding if the number of times is equal to a preset picture group size.

[0194] Fig.11 The structure diagram of the fourth embodiment of the video data processing device provided in the embodiment of the present application is as follows: Fig.11 As shown, based on the above embodiments, the first generating module 803 includes:

[0195] The division unit 1101 is used to divide the video data into a plurality of pixel blocks.

[0196] The searching unit 1102 is used to search for each pixel block in the reference frame based on a preset search range to obtain a matching block corresponding to the pixel block.

[0197] The determining unit 1103 is configured to determine, based on the pixel block, a position offset of a matching block corresponding to the pixel block relative to the pixel block.

[0198] The reconstruction unit 1104 is used to reconstruct the prediction block of the pixel block in the reference frame according to the position offset, and determine the residual block between the prediction block and the pixel block.

[0199] The assembling unit 1105 is used to encode and assemble the position offset and residual block corresponding to each pixel block to obtain a predicted frame.

[0200] Fig.12 The structure diagram of the fifth embodiment of the apparatus for processing video data provided in the embodiment of the present application is as follows: Fig.12 As shown, based on the above embodiments, the second generating module 901 specifically includes:

[0201] The division unit 1201 is used to divide the video data into a plurality of pixel blocks.

[0202] The first determining unit 1202 is configured to determine, for each pixel block, a predicted value of the pixel block according to pixel values ​​of neighboring pixel blocks of the pixel block.

[0203] The second determining unit 1203 is configured to determine a residual block according to the pixel values ​​and the prediction values ​​of the pixel block.

[0204] The assembling unit 1204 is used to transform, quantize and encode the residual block of each pixel block, and assemble the encoded information to obtain a key frame.

[0205] Fig.13 The structure diagram of the sixth embodiment of the apparatus for processing video data provided in the embodiment of the present application is as follows: Fig.13 As shown, based on the above embodiments, the video data processing device 800 further includes:

[0206] The first wake-up module 1301 is used to wake up the video recording device when a preset wake-up period is reached.

[0207] or;

[0208] The second wake-up module 1302 is used to wake up the video recording device when a preset target moving object and / or target sound is detected.

[0209] The apparatus for processing video data provided in the above embodiment is used to execute the method for processing video data in any of the above method embodiments. Its implementation principle and technical effect are similar and will not be described in detail here.

[0210] Fig.14 A schematic diagram of the structure of the video recording device provided in the embodiment of the present application is shown in FIG. Fig.14 As shown, the video recording device 1400 includes: a processor 1402, a memory 1401 communicatively connected to the processor 1402, a communication interface 1403, and a camera 1404;

[0211] The memory 1401 stores computer executable instructions;

[0212] The processor 1402 executes the computer-executable instructions stored in the memory 1401 to implement the method for processing video data in any method embodiment.

[0213] The communication interface 1403 is used for communication with external devices.

[0214] The camera 1404 is used to collect video data when the video recording device wakes up.

[0215] The embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to execute the video data processing methods provided by the various embodiments described above.

[0216] The computer-readable storage medium mentioned above can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0217] Optionally, a readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0218] An embodiment of the present application also provides a computer program product, which includes a computer program. The computer program is stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, the technical solution provided by any of the above method embodiments can be implemented.

[0219] In the present application, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship; in the formula, the character " / " indicates that the previous and next associated objects are in a "division" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0220] It is to be understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of description and are not intended to limit the scope of the embodiments of the present application. In the embodiments of the present application, the size of the sequence number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0221] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0222] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for processing video data, characterized in that: Applied to a video recording device, the method comprises: When the video recording device wakes up, the encoder state information and reference frame stored in the last wake-up are obtained; Reloading the encoder according to the encoder status information; Based on the encoder state information, if it is determined that the encoding type of this wake-up is inter-frame prediction encoding, inter-frame encoding is performed on the video data collected by this wake-up according to the reference frame to generate a prediction frame; The YUV data corresponding to the predicted frame is set as a new reference frame, and the encoder state information of the encoder after the current encoding is set as the new encoder state information; The new reference frame and the new encoder state information are stored in a non-volatile memory, and the system is powered off and put into sleep mode.

2. The method according to claim 1, characterized in that The method further comprises: Based on the encoder state information, if it is determined that the encoding type of this wake-up is intra-frame prediction encoding, intra-frame encoding is performed on the video data to generate a key frame; The YUV data corresponding to the key frame is set as the new reference frame.

3. The method according to claim 2, characterized in that The method further comprises: After the encoder is reloaded, the number of inter-frame prediction encodings between the last intra-frame prediction encoding and the current wake-up is obtained; If the number is less than the preset picture group size, it is determined that the encoding type of this wake-up is inter-frame prediction encoding.

4. The method according to claim 3, characterized in that The method further comprises: If the number is equal to the picture group size, it is determined that the encoding type of this wake-up is intra-frame prediction encoding.

5. The method according to claim 1, characterized in that If it is determined based on the encoder state information that the encoding type of this wake-up is inter-frame prediction encoding, inter-frame encoding is performed on the video data collected by this wake-up according to the reference frame to generate a prediction frame, including: Dividing the video data into a plurality of pixel blocks; For each pixel block, based on a preset search range, searching in the reference frame to obtain a matching block corresponding to the pixel block; Based on the pixel block, determining a position offset of a matching block corresponding to the pixel block relative to the pixel block; Reconstructing a prediction block of the pixel block in the reference frame according to the position offset, and determining a residual block between the prediction block and the pixel block; The position offset and residual block corresponding to each pixel block are encoded and assembled to obtain the predicted frame.

6. The method according to claim 2, characterized in that If it is determined based on the encoder state information that the encoding type of this wake-up is intra-frame prediction encoding, intra-frame encoding is performed on the video data to generate a key frame, including: Dividing the video data into a plurality of pixel blocks; For each pixel block, determining a predicted value of the pixel block according to pixel values ​​of adjacent pixel blocks of the pixel block; Determining a residual block according to the pixel value of the pixel block and the prediction value; The residual block of each pixel block is transformed, quantized and encoded, and the encoded information is assembled to obtain the key frame.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: When a preset wake-up period is reached, the video recording device wakes up; or; When a preset target moving object and / or target sound is detected, the video recording device wakes up.

8. A video data processing device, characterized in that: include: A first acquisition module, used for acquiring, when the video recording device wakes up, encoder state information and reference frames stored in the last wake-up; A reload module, used for reloading the encoder according to the encoder status information; A first generating module is used for, based on the encoder state information, if it is determined that the encoding type of this wake-up is inter-frame prediction encoding, then inter-frame encoding is performed on the video data collected by this wake-up according to the reference frame to generate a prediction frame; A first updating module, used to set the YUV data corresponding to the predicted frame as a new reference frame, and set the encoder state information of the encoder after the current encoding as the new encoder state information; The storage module is used to store the new reference frame and the new encoder state information in a non-volatile memory and power off and enter sleep mode.

9. A video recording device, characterized in that: include: A processor, and a memory, a communication interface, and a camera that are communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method for processing video data according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method for processing video data according to any one of claims 1 to 7.

Images