Video data processing method, device, equipment and storage medium

Abstract

The application provides a video recording data processing method, device, equipment and storage medium. It is applied to the field of video coding technology. The method comprises the following steps: when the video recording device is woken up, the last time the encoder state information and the reference frame stored when the device is woken up are obtained, the encoder is reloaded according to the encoder state information, if it is determined that the coding type is inter-frame prediction coding based on the encoder state information, the video recording data collected in the current wake-up is inter-frame coded according to the reference frame, a prediction frame is generated, the YUV data corresponding to the prediction frame is set as a new reference frame, the encoder state information of the encoder after the current coding is set as new encoder state information, the new reference frame and the new encoder state information are stored in the non-volatile memory, and the device is powered off and hibernated. Through the above method, the compression rate is improved, the energy consumption is reduced, the coding efficiency is improved, and the video quality is improved through flexible reference frame management in the low-power full-time video recording scene.

Description

Technical Field

[0001] The present application relates to the field of video coding technology, and in particular to a method, apparatus, device, and storage medium for processing video data. Background Art

[0002] Full-time recording in low-power mode is an advanced surveillance technology that significantly reduces energy consumption while enabling 24 / 7 recording. Low-power technology allows surveillance devices to remain in a dormant or low-power state most of the time, waking up and starting recording only when specific events are detected.

[0003] When the video recording device wakes up, the video recording device collects data, then performs intra-frame encoding on the collected data and saves the final video code stream data.

[0004] However, the video code stream data rate obtained by the above method is relatively high, which not only increases the storage and transmission burden, but also increases power consumption. Summary of the Invention

[0005] The present application provides a method, apparatus, device and storage medium for processing video data, which is used to solve the problem that the video code stream data rate obtained by encoding video data in the prior art is relatively high, which not only increases the storage and transmission burden but also increases power consumption.

[0006] In a first aspect, the present application provides a method for processing video data, applied to a video recording device, the method comprising:

[0007] When the video recording device wakes up, obtaining the encoder state information and reference frame stored during the last wake-up;

[0008] reloading the encoder according to the encoder status information;

[0009] Based on the encoder state information, if it is determined that the encoding type of this wake-up is inter-frame prediction encoding, inter-frame encoding is performed on the video data collected during this wake-up according to the reference frame to generate a prediction frame;

[0010] The YUV data corresponding to the predicted frame is set as a new reference frame, and the encoder state information of the encoder after the current encoding is set as the new encoder state information;

[0011] The new reference frame and the new encoder state information are stored in a non-volatile memory, and the system is powered off and put into sleep mode.

[0012] In combination with the first aspect, in some embodiments, the method further includes:

[0013] If it is determined based on the encoder state information that the encoding type for this wakeup is intra-frame prediction encoding, intra-frame encoding is performed on the video data to generate a key frame;

[0014] The YUV data corresponding to the key frame is set as the new reference frame.

[0015] In combination with the first aspect, in some embodiments, the method further includes:

[0016] After the encoder completes reloading, the number of inter-frame prediction encodings between the last intra-frame prediction encoding and the current wake-up is obtained;

[0017] If the number is less than the preset picture group size, it is determined that the encoding type of this wake-up is inter-frame prediction encoding.

[0018] In combination with the first aspect, in some embodiments, the method further includes:

[0019] If the number is equal to the preset picture group size, it is determined that the encoding type of this wake-up is intra-frame prediction encoding.

[0020] In combination with the first aspect, in some embodiments, if it is determined based on the encoder state information that the encoding type of this wakeup is inter-frame prediction encoding, inter-frame encoding is performed on the video data collected during this wakeup according to the reference frame to generate a predicted frame, including:

[0021] Dividing the video data into a plurality of pixel blocks;

[0022] For each pixel block, based on a preset search range, searching in the reference frame to obtain a matching block corresponding to the pixel block;

[0023] Based on the pixel block, determining a position offset of a matching block corresponding to the pixel block relative to the pixel block;

[0024] reconstructing a prediction block of the pixel block in the reference frame according to the position offset, and determining a residual block between the prediction block and the pixel block;

[0025] The position offset and residual block corresponding to each pixel block are encoded and assembled to obtain the predicted frame.

[0026] In combination with the first aspect, in some embodiments, if it is determined based on the encoder state information that the encoding type for this wakeup is intra-frame prediction encoding, intra-frame encoding is performed on the video data to generate a key frame, including:

[0027] Dividing the video data into a plurality of pixel blocks;

[0028] For each pixel block, determining a predicted value of the pixel block according to pixel values ​​of adjacent pixel blocks of the pixel block;

[0029] Determining a residual block according to the pixel value of the pixel block and the predicted value;

[0030] The residual block of each pixel block is transformed, quantized and encoded, and the encoded information is assembled to obtain the key frame.

[0031] In combination with the first aspect, in some embodiments, the method further includes:

[0032] When a preset wake-up period is reached, the video recording device wakes up;

[0033] or;

[0034] When a preset target moving object and / or target sound is detected, the video recording device wakes up.

[0035] In a second aspect, the present application provides a device for processing video data, comprising:

[0036] A first acquisition module is used to acquire the encoder state information and reference frame stored in the last wake-up when the video recording device wakes up;

[0037] A reloading module, configured to reload the encoder according to the encoder status information;

[0038] A first generating module is configured to, based on the encoder state information, if it is determined that the encoding type of this wake-up is inter-frame prediction encoding, perform inter-frame encoding on the video data collected during this wake-up according to the reference frame to generate a prediction frame;

[0039] A first updating module is configured to set the YUV data corresponding to the predicted frame as a new reference frame, and set the encoder state information of the encoder after the current encoding as the new encoder state information;

[0040] The storage module is used to store the new reference frame and the new encoder state information in a non-volatile memory and power off to sleep.

[0041] In conjunction with the second aspect, in some embodiments, the apparatus further includes:

[0042] A second generating module is configured to, based on the encoder state information, perform intra-frame encoding on the video data to generate a key frame if it is determined that the encoding type for this wakeup is intra-frame prediction encoding;

[0043] The second updating module is configured to set the YUV data corresponding to the key frame as the new reference frame.

[0044] In conjunction with the second aspect, in some embodiments, the apparatus further includes:

[0045] The second acquisition module is used to obtain the number of inter-frame prediction encodings between the last intra-frame prediction encoding and the current wake-up after the encoder completes reloading;

[0046] The first determining module is configured to determine that the encoding type for this wakeup is inter-frame prediction encoding if the number of times is less than a preset picture group size.

[0047] In conjunction with the second aspect, in some embodiments, the apparatus further includes:

[0048] The second determining module is configured to determine that the encoding type for this wakeup is intra-frame prediction encoding if the number of times is equal to a preset picture group size.

[0049] In conjunction with the second aspect, in some embodiments, the first generating module includes:

[0050] a dividing unit, configured to divide the video data into a plurality of pixel blocks;

[0051] a searching unit, configured to search for each pixel block in the reference frame based on a preset search range to obtain a matching block corresponding to the pixel block;

[0052] a determining unit, configured to determine, based on the pixel block, a position offset of a matching block corresponding to the pixel block relative to the pixel block;

[0053] a reconstruction unit, configured to reconstruct a prediction block of the pixel block in the reference frame according to the position offset, and determine a residual block between the prediction block and the pixel block;

[0054] The assembling unit is used to encode and assemble the position offset and residual block corresponding to each pixel block to obtain the predicted frame.

[0055] In conjunction with the second aspect, in some embodiments, the second generating module includes:

[0056] a dividing unit, configured to divide the video data into a plurality of pixel blocks;

[0057] A first determining unit is configured to determine, for each pixel block, a predicted value of the pixel block according to pixel values ​​of adjacent pixel blocks of the pixel block;

[0058] a second determining unit, configured to determine a residual block according to the pixel values ​​of the pixel block and the predicted value;

[0059] The assembling unit is used to transform, quantize and encode the residual block of each pixel block, and assemble the encoded information to obtain the key frame.

[0060] In conjunction with the second aspect, in some embodiments, the apparatus further includes:

[0061] A first wake-up module, configured to wake up the video recording device when a preset wake-up period is reached;

[0062] or;

[0063] The second wake-up module is used to wake up the video recording device when a preset target moving object and / or target sound is detected.

[0064] In a third aspect, the present application provides a video recording device, comprising: a processor, and a memory, a communication interface, and a camera in communication with the processor;

[0065] The memory stores computer-executable instructions;

[0066] The processor executes the computer-executable instructions stored in the memory to implement the method for processing video data according to any one of the first aspects.

[0067] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method for processing video data described in any of the aforementioned aspects.

[0068] In a fifth aspect, the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the method for processing video data described in any of the above aspects.

[0069] The present application provides a method, apparatus, device, and storage medium for processing video data. When the video recording device wakes up, it obtains the encoder status information and reference frame stored during the previous wakeup, and reloads the encoder based on the encoder status information. Based on the encoder status information, if the encoding type is determined to be inter-frame prediction encoding, the video data collected during this wakeup is inter-frame encoded based on the reference frame to generate a prediction frame. The YUV data corresponding to the prediction frame is set as a new reference frame, the encoder status information of the encoder after this encoding is set as the new encoder status information, the new reference frame and the new encoder status information are stored in a non-volatile memory, and the device is powered off and put into sleep mode. Through the above method, the compression rate is improved, energy consumption is reduced, and encoding efficiency is improved in low-power full-time recording scenarios. Flexible reference frame management also improves video quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0071] Figure 1 This is a diagram of an application scenario of the method for processing video data provided in an embodiment of the present application;

[0072] Figure 2 A flowchart of a first embodiment of a method for processing video data provided in an embodiment of the present application;

[0073] Figure 3 A flowchart of a second embodiment of a method for processing video data provided in an embodiment of the present application;

[0074] Figure 4 A flowchart of a third embodiment of the method for processing video data provided in an embodiment of the present application;

[0075] Figure 5 A flowchart of a fourth embodiment of a method for processing video data provided in an embodiment of the present application;

[0076] Figure 6 A flowchart of a fifth embodiment of a method for processing video data provided in an embodiment of the present application;

[0077] Figure 7 A flowchart of a specific example of a method for processing video data provided in an embodiment of the present application;

[0078] Figure 8 A schematic diagram of the structure of a first embodiment of a device for processing video data provided in an embodiment of the present application;

[0079] Figure 9 A schematic diagram of the structure of a second embodiment of a device for processing video data provided in an embodiment of the present application;

[0080] Figure 10 A schematic diagram of the structure of a third embodiment of a device for processing video data provided in an embodiment of the present application;

[0081] Figure 11 A schematic diagram of the structure of a fourth embodiment of a device for processing video data provided in an embodiment of the present application;

[0082] Figure 12 A schematic diagram of the structure of a fifth embodiment of a device for processing video data provided in an embodiment of the present application;

[0083] Figure 13 A schematic diagram of the structure of a sixth embodiment of a device for processing video data provided in an embodiment of the present application;

[0084] Figure 14 A schematic diagram of the structure of the video recording device provided in an embodiment of the present application.

[0085] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0086] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0087] Full-time recording in low-power mode is an advanced surveillance technology that significantly reduces energy consumption while enabling 24 / 7 recording. Low-power technology allows surveillance devices to remain in a dormant or low-power state most of the time, waking up and starting recording only when specific events are detected. While awake, the device collects data, performs intra-frame encoding on it, and saves the resulting video stream. This method is relatively simple and has no special hardware requirements. However, because all frames are intra-frame predictive coded, the resulting video stream has a higher bitrate, which increases storage and transmission burdens and power consumption.

[0088] To address the above-mentioned issues, the present application provides a method, apparatus, device, and storage medium for processing video data, thereby reducing the bitrate of video stream data, ensuring low power consumption, and reducing memory usage. Specifically, in the prior art, after the device wakes up, intra-frame prediction coding is performed on the captured frames to generate key frames. This method is simple and does not require any special hardware. However, because all frames are intra-frame prediction coded, the resulting stream data is large. If, after the device wakes up, it only captures data without encoding, and directly saves the captured raw data, and after multiple wake-ups, the accumulated saved data reaches a certain amount, all the saved data is encoded. However, this method saves a large amount of raw data and consumes a large amount of storage space. To address these issues, the inventors investigated whether it is possible to preset picture groups and reduce the bitrate of the video stream through hybrid coding to ensure low power consumption. Furthermore, during each encoding process, the YUV data of a frame of data encoded in that encoding is used as a reference frame for the next encoding. After each encoding is completed, the reference frame is deleted and replaced, thereby reducing storage resource consumption. Based on this, the technical solution of the present application is proposed.

[0089] Figure 1 This is an application scenario diagram of the method for processing video data provided in the embodiment of the present application, such as Figure 1 As shown, this scenario includes at least a recording device and a remote device, where the remote device can be a server, cloud, smartphone, laptop, or other electronic device. The recording device is in a low-power sleep state for a long time. The user can set a wake-up method. When the wake-up conditions are met, the recording device automatically wakes up to record, collects the video data, and then transmits the video stream to the remote device through hybrid encoding when the video stream reaches a certain amount or at a fixed time. The user can then view the recording information on the remote device.

[0090] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0091] Figure 2 The flowchart of the first embodiment of the method for processing video data provided in the embodiment of the present application is as follows: Figure 2 As shown, the method can be used for a video recording device, and the method specifically includes:

[0092] S201: When the video recording device wakes up, the encoder state information and reference frame stored during the last wake-up are obtained.

[0093] S202: Reload the encoder according to the encoder status information.

[0094] Low-power devices cannot be powered on all the time due to battery life considerations. Therefore, full-time recording for low-power devices is generally implemented by powering on periodically to encode a frame of data, then powering off, and then powering on again to encode another frame in the next period. However, since information such as the status and reference frame stored in the encoder is lost upon power failure, the encoder must be reset each time it is powered on. To reduce the bitrate of the encoded stream data, the status information and reference frame after each encoding can be stored in non-volatile memory. When the recording device wakes up, it retrieves the encoder status information and reference frame stored during the previous wakeup, and then reloads the encoder based on the encoder status information.

[0095] S203: Based on the encoder state information, if it is determined that the encoding type of this wakeup is inter-frame prediction encoding, inter-frame encoding is performed on the video data collected in this wakeup according to the reference frame to generate a prediction frame.

[0096] In this step, in order to reduce the bit rate of the encoded video stream, a hybrid encoding method can be used to ensure low power consumption. The video recording device is in a low-power standby state most of the time and wakes up under specific conditions. After the video recording device wakes up and completes the encoder reload, the video recording device collects the video data and then determines the encoding type of this encoding. If it is determined that the encoding type of this encoding is inter-frame prediction encoding, the reference frame stored in the last wake-up is obtained, and then the video data is inter-frame encoded to obtain a predicted frame.

[0097] Specifically, the video data is divided into multiple pixel blocks. For each pixel block, based on a preset search range, a search is performed in a reference frame to obtain a matching block corresponding to the pixel block. Based on the pixel block, a position offset of the matching block corresponding to the pixel block relative to the pixel block is determined. According to the position offset, a prediction block of the pixel block is reconstructed in the reference frame, and a residual block between the prediction block and the pixel block is determined. The position offset and residual block corresponding to each pixel block are encoded and assembled to obtain a prediction frame.

[0098] Optionally, when the recording device is in sleep mode, it will be awakened if any of the following conditions are met:

[0099] 1. When the preset wake-up period is reached, the recording device wakes up.

[0100] For example, the recording device automatically wakes up to collect data according to a preset schedule. The user can set the recording device to automatically wake up and start collecting data at a specific time (such as early morning or daytime every day), and automatically enter sleep mode after the preset time, or set it to automatically wake up and collect data every 2 hours, and automatically enter sleep mode after 10 minutes.

[0101] 2. When a preset target moving object and / or target sound is detected, the recording device wakes up.

[0102] For example, the recording device may have an integrated motion detection sensor or algorithm that can monitor changes in motion within the monitoring area in real time. When a certain level of motion is detected within the preset area (e.g., a pixel change rate exceeding a threshold), the recording device automatically wakes up from a low-power standby state and begins collecting data.

[0103] You can also rely on sound sensors to detect changes in ambient sound. When the sound sensor detects sounds above a preset decibel, it triggers the video recording device to wake up and start collecting data.

[0104] In one possible implementation, the user can wake up the recording device through remote commands or an application. Once the recording device is connected to the internet or a local area network, the user can send a wake-up command through a mobile app, a webpage, or other remote control. Upon receiving the command, the recording device automatically wakes up and begins collecting data.

[0105] Alternatively, the device can automatically wake up based on changes in environmental parameters (such as light or temperature). The recording device is equipped with a corresponding environmental sensor. When it detects that the environmental parameter reaches a preset threshold (such as a sudden decrease in light or a sharp increase in temperature), the recording device is triggered to wake up and start collecting data.

[0106] Alternatively, the device can automatically wake up based on specific events. This can include smoke detectors (fires) or intruder detectors (security systems). When these specific events occur, the recording device automatically wakes up and begins collecting data.

[0107] S204: Set the YUV data corresponding to the predicted frame as a new reference frame, and set the encoder state information of the encoder after the current encoding as the new encoder state information.

[0108] S205: Store the new reference frame and the new encoder state information in the non-volatile memory, and power off and enter sleep mode.

[0109] In this step, after completing the inter-frame prediction coding of the video data, in order to save storage space and thus improve coding efficiency, the reference frame stored in the last wake-up can be deleted, so that the YUV data of the original data corresponding to the video data encoded this time is set as a new reference frame for preservation, and then used for the next wake-up coding. In order for the next recording device to be powered on continuously with the last power-on, the encoder status information after this encoding is stored as the new status information and the new reference frame in the non-volatile memory, and then powered off and put into sleep until the next wake-up.

[0110] Specifically, after the encoding is completed, the video data collected this time is copied to the memory every day, and then converted into YUV format data and stored in the memory as a new reference frame, and the original reference frame is deleted. The memory is a memory that does not lose data when power is lost, such as a disk.

[0111] The method for processing recorded data provided in this embodiment retrieves the encoder status information and reference frame stored during the previous wakeup when the recording device wakes up. Based on the encoder status information, the encoder is reloaded. If the encoding type is determined to be inter-frame prediction based on the encoder status information, inter-frame encoding is performed on the recorded data collected during the current wakeup based on the reference frame to generate a predicted frame. The YUV data corresponding to the predicted frame is set as the new reference frame. The encoder status information of the encoder after the current encoding is set as the new encoder status information. The new reference frame and new encoder status information are stored in non-volatile memory, and the device is powered off and put into sleep mode. This method improves compression ratio, reduces energy consumption, and improves encoding efficiency in low-power, full-time recording scenarios. Flexible reference frame management also enhances video quality.

[0112] Figure 3 The flowchart of the second embodiment of the method for processing video data provided in the embodiment of the present application is as follows: Figure 3 As shown, based on the above embodiment, the method further includes:

[0113] S301: Based on the encoder state information, if it is determined that the encoding type of this wakeup is intra-frame prediction encoding, intra-frame encoding is performed on the video data to generate key frames.

[0114] In this step, in order to reduce the bit rate of the encoded video stream, a hybrid encoding method can be used to ensure low power consumption. The recording device is in a low-power standby state most of the time and wakes up under specific conditions. When the recording device wakes up, the recording device collects the recording data and then determines the encoding type of this encoding. If the encoding type is determined to be intra-frame prediction encoding, the recording data is directly intra-frame encoded to generate key frames.

[0115] Specifically, the video data is divided into multiple pixel blocks. For each pixel block, the predicted value of the pixel block is determined according to the pixel values ​​of the adjacent pixel blocks of the pixel block. The residual block is determined according to the pixel value and the predicted value of the pixel block. The residual block of each pixel block is transformed, quantized and encoded, and the encoded information is assembled to obtain a key frame.

[0116] S302: Set the YUV data corresponding to the key frame as a new reference frame.

[0117] In this step, after the current encoding is completed, in order to save storage space and improve encoding efficiency, the reference frame stored in the last wake-up is deleted, and the YUV data corresponding to the key frame obtained in this encoding is set as a new reference frame.

[0118] It should be noted that if the recording device is awakened for the first time, there is no reference frame stored in the memory, so there is no need to delete the reference frame, and the YUV data corresponding to the key frame obtained by this encoding is directly set as the reference frame.

[0119] The method for processing recorded video data provided in this embodiment, if the encoding type is determined to be intra-frame predictive coding, performs intra-frame encoding on the recorded data, generates a key frame, deletes the reference frame, and sets the YUV data corresponding to the key frame as the new reference frame. This method improves compression ratio, reduces transmission bandwidth and storage space, and enhances decoding efficiency, error resilience, flexibility, and adaptability in full-time recording scenarios in low-power mode.

[0120] Figure 4 This is a flow chart of Example 3 of the method for processing video data provided in the embodiment of the present application, as shown in FIG. Figure 4As shown, based on the above embodiment, the method further includes:

[0121] S401: After the encoder completes reloading, the number of inter-frame prediction encodings between the last intra-frame prediction encoding and the current wake-up is obtained.

[0122] S402: If the number is less than the preset GOP size, determining that the encoding type for this wakeup is inter-frame prediction encoding.

[0123] S403: If the number is equal to the preset GOP size, determining that the encoding type for this wakeup is intra-frame prediction encoding.

[0124] After the recording device wakes up and the encoder is reloaded, it captures the recorded data. To determine the encoding type for this encoding, the device obtains the number of inter-frame prediction encodings between the last intra-frame prediction encoding and the current wakeup, and compares this number with the pre-set GOP size. A GOP is an encoding method for a group of consecutive frames. Typically, a GOP begins with a keyframe, followed by a series of non-keyframes (predicted frames) until the next keyframe.

[0125] If the number is less than the preset GOP size, inter-frame prediction coding can continue, reducing the frequency of keyframe generation. The encoding type for this wakeup is determined to be inter-frame prediction coding. If the number is equal to the GOP size, it means that a sufficient number of non-keyframes have been encoded since the last keyframe. By inserting a new keyframe, the image quality in the video stream is updated promptly and can better adapt to the encoding of subsequent frames. Furthermore, inserting a keyframe when the GOP size is reached can improve the error resilience of the video stream. The encoding type for this wakeup is determined to be intra-frame prediction coding.

[0126] The method for processing video data provided in this embodiment obtains the number of inter-frame prediction codes performed between the last intra-frame prediction code and the current wake-up when the video recording device wakes up. If the number is less than the preset GOP size, the coding type for the current wake-up is determined to be inter-frame prediction code. If the number is equal to the GOP size, the coding type for the current wake-up is determined to be intra-frame prediction code. By comparing the number of inter-frame prediction codes with the preset GOP size, the coding strategy is automatically adjusted, thereby meeting the video recording device's requirement for low energy consumption. By using the preset GOP size as a threshold, the generation of key frames can be controlled while maintaining a certain compression efficiency. This helps to find a balance between compression efficiency and energy consumption, without excessive energy consumption or excessive sacrifice of video quality.

[0127] Figure 5 This is a flow chart of a fourth embodiment of the method for processing video data provided in the present application, as shown in FIG. Figure 5As shown, based on the above embodiments, step S203 specifically includes:

[0128] S501: Divide the video data into a plurality of pixel blocks.

[0129] In this step, the recorded data (i.e., video frames) is divided into multiple pixel blocks (commonly also referred to as macroblocks, coding units, or prediction units). The size of these blocks can be fixed or adaptive, depending on the coding standard and the characteristics of the video content. For example, 16x16 pixel, 8x8 pixel, or smaller 4x4 pixel blocks. This is not specifically limited in the present embodiment.

[0130] S502: For each pixel block, based on a preset search range, search in the reference frame to obtain a matching block corresponding to the pixel block.

[0131] In this step, a search range is set for each pixel block, typically near the corresponding position in the previous frame (or frames) before the current frame. For each pixel block, a matching criterion (such as minimum mean square error (MSE) or minimum mean absolute error (MAD)) is used within the preset search range to find the most similar matching block in the reference frame.

[0132] For example, a search algorithm is used to find the best matching block within the search range. The search algorithm can be a full search (FS), a fast search (such as a diamond search, a hexagonal search, etc.), or other more efficient algorithms. This embodiment of the present application is not specifically limited.

[0133] S503: Based on the pixel block, determine a position offset of a matching block corresponding to the pixel block relative to the pixel block.

[0134] In this step, after obtaining the matching block corresponding to each pixel block, the position offset of the matching block relative to the current pixel block is calculated.

[0135] For example, assume that the coordinate position of the current pixel block in the reference frame is (x1, y1). It should be noted that the coordinates here can be offsets relative to a reference point, and the coordinate position of the best matching block in the reference frame is (x2, y2).

[0136] Then, the horizontal and vertical components of the position offset can be calculated by the following formulas respectively:

[0137] MVx=x2-x1

[0138] MVy=y2-y1

[0139] Wherein, MVx represents the horizontal component of the position offset, and MVy represents the vertical component of the position offset.

[0140] S504: reconstructing a prediction block of the pixel block in the reference frame according to the position offset, and determining a residual block between the prediction block and the pixel block.

[0141] In this step, after the position offset corresponding to each pixel block is obtained, the prediction block of each pixel block is reconstructed in the reference frame according to the position offset of each pixel block, and then the residual block is calculated.

[0142] For example, for each pixel block, if the position offset of the pixel block is not an integer, that is, there is sub-pixel precision, then interpolation is required in the reference frame to generate pixel values ​​at the sub-pixel position, which can be achieved through an interpolation algorithm (such as bilinear interpolation, bicubic interpolation, etc.). If the position offset of the pixel block is an integer and there is no sub-pixel precision requirement, the pixel block at the corresponding position can be directly copied from the reference frame as the prediction block. If there is sub-pixel precision, after interpolating the pixel values ​​at the sub-pixel position, the prediction block is generated based on these values ​​or may also include pixel values ​​at integer positions.

[0143] Optionally, to further improve the quality of the predicted block, it can be post-processed, for example, by performing operations such as denoising, sharpening, and color correction.

[0144] Finally, the pixel difference between the current pixel block and its predicted block is calculated to obtain the residual block, which also contains information about the unpredicted part of the current pixel block.

[0145] S505: Encode and assemble the position offset and residual block corresponding to each pixel block to obtain a predicted frame.

[0146] In this step, the position offset and residual block of each pixel block are encoded. The encoding process may include transformation (such as DCT), quantization, and entropy coding to further reduce the amount of data. The encoded position offset and residual block information are then assembled to form a predicted frame.

[0147] The method for processing video data provided in this embodiment divides the video data into multiple pixel blocks. For each pixel block, a search is performed within a reference frame based on a preset search range to obtain a matching block corresponding to the pixel block. Based on the pixel block, the position offset of the matching block relative to the pixel block is determined. Based on the position offset, a prediction block for the pixel block is reconstructed in the reference frame, and a residual block between the prediction block and the pixel block is determined. The position offset and residual block corresponding to each pixel block are encoded and assembled to obtain a predicted frame. This method eliminates temporal redundancy, reduces data volume, and achieves a high compression rate while ensuring image quality.

[0148] Figure 6 This is a flow chart of a fifth embodiment of the method for processing video data provided in the embodiment of the present application, as shown in FIG. Figure 6 As shown, based on the above embodiments, step S301 specifically includes:

[0149] S601: Divide the video data into a plurality of pixel blocks.

[0150] The specific implementation of this step is the same as that of step S501 in the above embodiment, and will not be repeated here.

[0151] S602: For each pixel block, determine a prediction value of the pixel block according to pixel values ​​of adjacent pixel blocks of the pixel block.

[0152] In this step, for each pixel block, the value of the current pixel block is predicted using information from adjacent pixel blocks, thereby reducing the amount of data required for encoding.

[0153] Specifically, an appropriate prediction mode is selected based on the coding standard. For example, in H.264 / AVC, luminance blocks can have multiple 4x4 and 16x16 prediction modes. The predicted value of the current pixel block is calculated based on the selected prediction mode using the pixel values ​​of adjacent pixel blocks (usually the pixel blocks to the left and above).

[0154] S603: Determine a residual block according to the pixel values ​​and the prediction values ​​of the pixel block.

[0155] For each pixel block, the difference between its actual value and the predicted value is calculated to obtain the residual block.

[0156] S604: transform, quantize and encode the residual block of each pixel block, assemble the encoded information to obtain a key frame.

[0157] In this step, the data volume of the residual block is further reduced through transformation and quantization, and is encoded for easy transmission and storage.

[0158] Specifically, the discrete cosine transform (DCT) is used to convert the residual block from the spatial domain to the frequency domain. This helps remove spatial correlations in the residual block. The transformed coefficients are quantized using a specific quantization step size to reduce the data's precision and dynamic range. The larger the quantization step size, the higher the compression ratio. The quantized coefficients are encoded (e.g., using entropy coding methods such as Huffman coding and arithmetic coding) to generate the final encoded data. The encoded information corresponding to each pixel block is then assembled to obtain the key frame.

[0159] The video data processing method provided in this embodiment divides the video data into multiple pixel blocks. For each pixel block, a predicted value is determined based on the pixel values ​​of its adjacent pixel blocks. A residual block is then determined based on the pixel values ​​and predicted values. The residual block for each pixel block is transformed, quantized, and encoded, and the encoded information is assembled to generate a key frame. This method, which performs intra-frame predictive coding based on the pixel values ​​of adjacent pixel blocks, eliminates spatial redundancy, improves compression efficiency, enhances image quality, and increases flexibility, facilitating decoding and error recovery.

[0160] Figure 7 A flowchart of a specific example of a method for processing video data provided in an embodiment of the present application is shown as follows: Figure 7 As shown, the method includes:

[0161] S701: The video recording device wakes up for the first time and performs intra-frame prediction coding.

[0162] The video recording device performs intra-frame prediction encoding when it is first awakened. The specific implementation process is the same as the intra-frame prediction encoding process in the aforementioned embodiment, and will not be repeated here.

[0163] S702: Save original data.

[0164] After the intra-frame prediction coding is completed, the YUV data of the original data of the key frame obtained by this coding is saved as a reference frame.

[0165] S703: Sleep.

[0166] S704: Wake up.

[0167] After completing one encoding, the video recording device enters a dormant state until a wake-up condition is met, and then collects video data after waking up. The wake-up condition is the same as in the above embodiment and will not be repeated here.

[0168] S705: Reload the encoder to check whether intra-frame prediction coding is currently required.

[0169] Since this is not the first wakeup, the encoder state information and reference frames stored in the non-volatile memory after the previous encoding are first obtained. Based on the pre-set GOP size, it is determined whether intra-frame prediction encoding is currently required. The specific determination process is the same as steps S401 to S403 in the aforementioned embodiment and will not be repeated here.

[0170] S706: If intra-frame prediction coding is not currently required, the previously saved original data is used as a reference frame.

[0171] If it is determined that intra-frame prediction coding is not required, inter-frame prediction coding is performed. Before coding, a reference frame needs to be obtained, and the reference frame saved by the first awakening is retrieved in advance.

[0172] S707: Inter-frame prediction coding.

[0173] The inter-frame prediction coding is performed based on the reference frame. The specific implementation process is the same as the inter-frame prediction coding process in the above embodiment, and will not be repeated here.

[0174] S708: If intra-frame prediction coding is currently required, intra-frame prediction coding is performed.

[0175] If it is determined that intra-frame prediction coding is currently required, intra-frame prediction coding is performed, and the specific coding process is not described in detail here.

[0176] S709: Delete the original data saved previously

[0177] S710: Save the original data of the current frame.

[0178] After the current encoding is completed, the last saved reference frame is deleted, the YUV data of the original data of the current frame is saved as a new reference frame, and then the system goes into sleep until the next wake-up.

[0179] The method for processing recorded data provided in this embodiment sets the GOP size and then determines the encoding type each time the recording device wakes up. After each encoding session, the YUV data of the original data is saved as a reference frame, and the previously saved reference frame is deleted. This hybrid encoding method not only reduces the bitrate of the video stream, but also improves compression, reduces transmission bandwidth, and reduces storage space while maintaining video quality.

[0180] Figure 8 This is a structural diagram of a first embodiment of a device for processing video data provided in an embodiment of the present application, as shown in FIG. Figure 8 As shown, the video data processing device 800 includes:

[0181] The first acquisition module 801 is configured to acquire the encoder state information and reference frame stored in the last wake-up when the video recording device wakes up.

[0182] The reloading module 802 is used to reload the encoder according to the encoder status information.

[0183] The first generating module 803 is configured to, based on the encoder state information, if the encoding type is determined to be inter-frame prediction encoding, perform inter-frame encoding on the video data collected during this wakeup according to the reference frame stored during the previous wakeup to generate a prediction frame.

[0184] The first updating module 804 is configured to set the YUV data corresponding to the predicted frame as a new reference frame, and set the encoder state information of the encoder after the current encoding as the new encoder state information.

[0185] The storage module 805 is used to store the new reference frame and the new encoder state information in the non-volatile memory and power off and enter sleep mode.

[0186] Figure 9 This is a structural diagram of a second embodiment of a device for processing video data provided in an embodiment of the present application, as shown in FIG. Figure 9 As shown, based on the above embodiment, the video data processing device 800 further includes:

[0187] The second generating module 901 is configured to perform intra-frame encoding on the video data to generate key frames if the encoding type is determined to be intra-frame prediction encoding based on the encoder state information.

[0188] The second updating module 902 is configured to set the YUV data corresponding to the key frame as a new reference frame.

[0189] Figure 10 This is a structural diagram of a third embodiment of the apparatus for processing video data provided in the embodiment of the present application, as shown in FIG. Figure 10 As shown, based on the above embodiment, the video data processing device 800 further includes:

[0190] The second acquisition module 1001 is configured to acquire the number of inter-frame prediction encodings between the last intra-frame prediction encoding and the current wake-up after the encoder completes reloading.

[0191] The first determining module 1002 is configured to determine that the encoding type for this wakeup is inter-frame prediction encoding if the number of times is less than a preset GOP size.

[0192] Optionally, the video data processing device 800 further includes:

[0193] The second determining module 1003 is configured to determine that the encoding type for this wakeup is intra-frame prediction encoding if the number of times is equal to a preset GOP size.

[0194] Figure 11 This is a structural diagram of a fourth embodiment of a device for processing video data provided in an embodiment of the present application, as shown in FIG. Figure 11 As shown, based on the above embodiments, the first generating module 803 includes:

[0195] The division unit 1101 is configured to divide the video data into a plurality of pixel blocks.

[0196] The searching unit 1102 is configured to search for each pixel block in the reference frame based on a preset search range to obtain a matching block corresponding to the pixel block.

[0197] The determining unit 1103 is configured to determine, based on the pixel block, a position offset of a matching block corresponding to the pixel block relative to the pixel block.

[0198] The reconstruction unit 1104 is configured to reconstruct a prediction block of the pixel block in the reference frame according to the position offset, and determine a residual block between the prediction block and the pixel block.

[0199] The assembling unit 1105 is configured to encode and assemble the position offset and residual block corresponding to each pixel block to obtain a predicted frame.

[0200] Figure 12 This is a structural diagram of a fifth embodiment of a device for processing video data provided in an embodiment of the present application, as shown in FIG. Figure 12 As shown, based on the above embodiments, the second generating module 901 specifically includes:

[0201] The division unit 1201 is configured to divide the video data into a plurality of pixel blocks.

[0202] The first determining unit 1202 is configured to determine, for each pixel block, a predicted value of the pixel block according to pixel values ​​of adjacent pixel blocks of the pixel block.

[0203] The second determining unit 1203 is configured to determine a residual block according to the pixel values ​​and the prediction values ​​of the pixel block.

[0204] The assembling unit 1204 is configured to transform, quantize, and encode the residual block of each pixel block, and assemble the encoded information to obtain a key frame.

[0205] Figure 13 This is a structural diagram of a sixth embodiment of a device for processing video data provided in an embodiment of the present application, as shown in FIG. Figure 13 As shown, based on the above embodiments, the video data processing device 800 further includes:

[0206] The first wake-up module 1301 is configured to wake up the video recording device when a preset wake-up period is reached.

[0207] or;

[0208] The second wake-up module 1302 is configured to wake up the video recording device when a preset target moving object and / or target sound is detected.

[0209] The video data processing device provided in the above embodiment is used to execute the video data processing method in any of the above method embodiments. Its implementation principles and technical effects are similar and will not be described in detail here.

[0210] Figure 14 A schematic diagram of the structure of the video recording device provided in the embodiment of the present application is shown as follows: Figure 14 As shown, the video recording device 1400 includes: a processor 1402, a memory 1401 communicatively connected to the processor 1402, a communication interface 1403, and a camera 1404;

[0211] Memory 1401 stores computer-executable instructions;

[0212] The processor 1402 executes the computer-executable instructions stored in the memory 1401 to implement the method for processing video data in any method embodiment.

[0213] The communication interface 1403 is used for communication with external devices.

[0214] The camera 1404 is used to collect video data when the video recording device wakes up.

[0215] An embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to execute the video data processing methods provided in the various embodiments described above.

[0216] The computer-readable storage medium mentioned above may be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic storage, flash memory, magnetic disk, or optical disk. The computer-readable storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0217] Optionally, a readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0218] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when at least one processor executes the computer program, it can implement the technical solution provided by any of the above method embodiments.

[0219] In this application, "at least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship; in the formula, the character " / " indicates that the previous and next associated objects are in a "division" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0220] It is understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of description and are not intended to limit the scope of the embodiments of the present application. In the embodiments of the present application, the order of the sequence numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0221] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0222] It should be understood that the present application is not limited to the exact structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for processing video data, characterized in that: Applied to a video recording device, the method comprises: When the video recording device wakes up, obtaining the encoder state information and reference frame stored during the last wake-up; reloading the encoder according to the encoder status information; Based on the encoder state information, if it is determined that the encoding type of this wake-up is inter-frame prediction encoding, inter-frame encoding is performed on the video data collected during this wake-up according to the reference frame to generate a prediction frame; The YUV data corresponding to the predicted frame is set as a new reference frame, and the encoder state information of the encoder after the current encoding is set as the new encoder state information; The new reference frame and the new encoder state information are stored in a non-volatile memory, and the system is powered off and put into sleep mode.

2. The method according to claim 1, characterized in that The method further comprises: If it is determined based on the encoder state information that the encoding type for this wakeup is intra-frame prediction encoding, intra-frame encoding is performed on the video data to generate a key frame; The YUV data corresponding to the key frame is set as the new reference frame.

3. The method according to claim 2, characterized in that The method further comprises: After the encoder completes reloading, the number of inter-frame prediction encodings between the last intra-frame prediction encoding and the current wake-up is obtained; If the number is less than the preset picture group size, it is determined that the encoding type of this wake-up is inter-frame prediction encoding.

4. The method according to claim 3, characterized in that The method further comprises: If the number is equal to the picture group size, it is determined that the encoding type of this wake-up is intra-frame prediction encoding.

5. The method according to claim 1, wherein If it is determined based on the encoder state information that the encoding type for this wakeup is inter-frame prediction encoding, inter-frame encoding is performed on the video data collected for this wakeup according to the reference frame to generate a prediction frame, including: Dividing the video data into a plurality of pixel blocks; For each pixel block, based on a preset search range, searching in the reference frame to obtain a matching block corresponding to the pixel block; Based on the pixel block, determining a position offset of a matching block corresponding to the pixel block relative to the pixel block; reconstructing a prediction block of the pixel block in the reference frame according to the position offset, and determining a residual block between the prediction block and the pixel block; The position offset and residual block corresponding to each pixel block are encoded and assembled to obtain the predicted frame.

6. The method according to claim 2, characterized in that If it is determined based on the encoder state information that the encoding type for this wakeup is intra-frame prediction encoding, intra-frame encoding is performed on the video data to generate a key frame, including: Dividing the video data into a plurality of pixel blocks; For each pixel block, determining a predicted value of the pixel block according to pixel values ​​of adjacent pixel blocks of the pixel block; Determining a residual block according to the pixel value of the pixel block and the predicted value; The residual block of each pixel block is transformed, quantized and encoded, and the encoded information is assembled to obtain the key frame.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: When a preset wake-up period is reached, the video recording device wakes up; or; When a preset target moving object and / or target sound is detected, the video recording device wakes up.

8. A video data processing device, characterized in that: include: The first acquisition module is used to acquire the encoder state information and reference frame stored in the last wake-up when the video recording device wakes up; A reloading module, configured to reload the encoder according to the encoder status information; A first generating module is configured to, based on the encoder state information, if it is determined that the encoding type of this wake-up is inter-frame prediction encoding, perform inter-frame encoding on the video data collected during this wake-up according to the reference frame to generate a prediction frame; A first updating module is configured to set the YUV data corresponding to the predicted frame as a new reference frame, and set the encoder state information of the encoder after the current encoding as the new encoder state information; The storage module is used to store the new reference frame and the new encoder state information in a non-volatile memory and power off to sleep.

9. A video recording device, characterized in that: include: A processor, and a memory, a communication interface, and a camera communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method for processing video data according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method for processing video data according to any one of claims 1 to 7.

Images