Motion perception video recording method and device based on monitoring video inter-frame decoding

Abstract

The invention discloses a motion perception video recording method and device based on monitoring video inter-frame decoding, and the method comprises the steps: decoding a monitoring video into a frame, and obtaining macro block information on a frame image; obtaining a macro block motion vector, a macro block residual error, the quantity of macro block non-zero transformation coefficients, a macro block quantization parameter and a macro block entropy decoding bit stream length according to macro block information on adjacent frame images, and determining a comprehensive motion sensing value of the frame images by combining respective weighting coefficients of the macro block motion vector, the macro block residual error, the macro block non-zero transformation coefficient quantity, the macro block quantization parameter and the macro block entropy decoding bit stream length; if the comprehensive motion sensing value of the frame image exceeds a set sensing threshold value, starting to record the monitoring video; and otherwise, stopping recording the monitoring video. According to the method, the image change is perceived and the movement of the object in the image is identified by monitoring the inter-frame decoding of the video, so that the video recording function is started, the efficient video compression is realized, the technical difficulty is reduced, and the storage resource is saved.

Description

Technical Field

[0001] The present invention relates to the technical field of surveillance video recording, and in particular to a motion perception recording method and device based on surveillance video inter-frame decoding. Background Art

[0002] As social security needs continue to grow, video surveillance systems have become an important tool for maintaining public security and protecting personal property. Traditional video surveillance systems usually use 24 / 7 recording, which ensures that no potential security incidents are missed, but also brings a huge storage burden. According to statistics, most surveillance videos in some scenarios are static images, and most static images are actually useless, which not only wastes storage resources, but also increases the difficulty of data management and retrieval.

[0003] The application of artificial intelligence (AI) technology in video surveillance has made significant progress, but there are still some limitations: high-performance AI technology requires strong computing resource support. When running on the server, the computing power is limited, which may cause recording delays or performance degradation. At the same time, high-quality AI technology requires a large amount of labeled data for training. The data acquisition and labeling process is time-consuming and costly. In addition, due to different application scenarios, the performance of AI technology may vary, and it also needs to be optimized and adjusted for specific scenarios, which greatly increases the recording technology requirements and energy consumption of motion surveillance videos. Summary of the invention

[0004] In response to the problems existing in the prior art, the present invention provides a motion perception recording method and device based on monitoring video inter-frame decoding, which perceives image changes through monitoring video inter-frame decoding and identifies the movement of objects in the image, thereby enabling the recording function, achieving efficient video compression, reducing technical difficulty, and saving storage resources.

[0005] To achieve the above technical objectives, the present invention adopts the following technical solution: a motion perception recording method based on monitoring video inter-frame decoding, specifically comprising the following steps:

[0006] Step 1: Decode the surveillance video into frames and obtain macroblock information on the frame image;

[0007] Step 2: According to the macroblock information on the adjacent frame images, the macroblock motion vector, the macroblock residual, the number of non-zero transform coefficients of the macroblock, the macroblock quantization parameter and the length of the macroblock entropy decoding bit stream are obtained, and the comprehensive motion perception value of the frame image is determined by combining the respective weighting coefficients;

[0008] Step 3: If the comprehensive motion perception value of the frame image exceeds the set perception threshold, start recording the surveillance video; otherwise, stop recording the surveillance video.

[0009] Furthermore, the comprehensive motion perception value of the frame image includes: a motion vector perception value of the frame image, a residual data perception value of the frame image, a non-zero transform coefficient perception value of the frame image, a quantization perception value of the frame image, and an entropy decoding perception value of the frame image.

[0010] Furthermore, the calculation process of the motion vector perception value of the frame image is: calculate the similarity between each macroblock in the current frame image and the macroblock in the previous frame image, find the macroblock that best matches each macroblock in the current frame image and the previous frame image according to the maximum value of the similarity, determine the macroblock motion vector of each macroblock in the current frame image relative to the macroblock that best matches the previous frame image, calculate the standard deviation Vmv of all macroblock motion vectors in the current frame image, and combine the weighting coefficient Wmv of all macroblock motion vectors to obtain the motion vector perception value Pmv=Vmv×Wmv of the current frame image.

[0011] Furthermore, the calculation process of the residual data perception value of the frame image is: calculate the offset of the pixel point in each macroblock in the current frame image and the corresponding pixel point in the most matching macroblock in the previous frame image according to the maximum value of the similarity, superimpose the offsets of all pixel points in each macroblock to obtain the macroblock residual, calculate the standard deviation Vrd of all macroblock residuals in the current frame image, and combine the weighting coefficient Wrd of all macroblock residuals to obtain the residual data perception value Prd = Vrd × Wrd of the current frame image.

[0012] Furthermore, the calculation process of the non-zero transform coefficient perception value of the frame image is: perform DCT transform on each macroblock in the current frame image, convert the pixel value in each macroblock into a frequency domain coefficient, count the number of non-zero transform coefficients of each macroblock in the current frame image, calculate the standard deviation Vcv of the number of non-zero transform coefficients of all macroblocks in the current frame image, and combine the weighted coefficient Wcv of the number of non-zero transform coefficients of all macroblocks to obtain the non-zero transform coefficient perception value Pcv=Vcv×Wcv of the current frame image.

[0013] Furthermore, the calculation process of the quantization perception value of the frame image is: according to the maximum similarity value, the change in the quantization parameter of each macroblock in the current frame image and the macroblock quantization parameter of the most matching macroblock in the previous frame image is counted, and the standard deviation Vqp of the change in the quantization parameter of all macroblocks in the current frame image is calculated, and the weighted coefficient Wqp of the change in the quantization parameter of all macroblocks is combined to obtain the quantization perception value Pqp = Vqp × Wqp of the current frame image.

[0014] Furthermore, the calculation process of the entropy decoding perception value of the frame image is: according to the maximum similarity value, the difference between the entropy decoding bit stream length of each macroblock in the current frame image and the macroblock entropy decoding bit stream length of the most matching macroblock in the previous frame image is calculated, and the standard deviation Vec of the entropy decoding bit stream length difference of all macroblocks in the current frame image is calculated. Combined with the weighted coefficient Wec of the entropy decoding bit stream length difference of all macroblocks, the entropy decoding perception value Pec=Vec×Wec of the current frame image is obtained.

[0015] Further, Wmv+Wrd+Wcv+Wqp+Wec=1.

[0016] Furthermore, step 3 includes the following sub-steps:

[0017] Step 3.1, collect several consecutive static frame images, obtain the macroblock information on each static frame image, repeat step 2, obtain the comprehensive motion perception value of each static frame image, calculate the average of the comprehensive motion perception values ​​of all static frame images, and determine the initial perception threshold in, represents the mean value of the comprehensive motion perception value of all static frame images, and w represents the perception threshold adjustment coefficient;

[0018] Step 3.2: If the comprehensive motion perception value of the frame image exceeds the initial perception threshold, start recording the surveillance video. If the comprehensive motion perception value of the frame images for one consecutive hour exceeds the perception threshold, use the average of the comprehensive motion perception values ​​of the frame images for one consecutive hour to update the perception threshold. If the comprehensive motion perception value of the frame images for 30 consecutive seconds is lower than the perception threshold, stop recording.

[0019] Furthermore, the present invention also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the motion perception recording method based on inter-frame decoding of surveillance video is implemented.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] (1) The motion perception recording method based on inter-frame decoding of surveillance video of the present invention uses macroblock motion vector, macroblock residual, number of non-zero transform coefficients of macroblock, macroblock quantization parameter and length of macroblock entropy decoding bit stream to calculate the comprehensive motion perception value of frame image. By analyzing the inter-frame decoding characteristics in the surveillance video, the recording function is started only when the frame image motion is detected, thereby improving the accuracy of frame image motion detection, realizing real-time frame image motion detection and recording start, having a fast response speed, avoiding the storage of a large number of useless static images, significantly reducing the storage of invalid video clips, saving storage resources, reducing storage costs, supporting multiple types of decoding standards, and being suitable for various real-time surveillance scenarios;

[0022] (2) The motion perception recording method based on inter-frame decoding of surveillance video of the present invention has a small amount of calculation, reduces the dependence on high-performance computing resources, and reduces energy consumption. In addition, since the calculation of the comprehensive motion perception value of the frame image in the present invention does not involve in-depth analysis of the specific content of the frame image, the privacy of the surveillance video is avoided from being violated;

[0023] (3) The present invention dynamically adjusts the perception threshold of the motion perception recording method based on inter-frame decoding of surveillance video, thereby further reducing the false alarm rate, and can perform adaptive optimization based on historical data and real-time feedback to improve the robustness of frame image motion detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 The present invention is a flow chart of a motion-aware recording method based on monitoring video inter-frame decoding. DETAILED DESCRIPTION

[0025] The technical solution of the present invention is further explained below in conjunction with the accompanying drawings.

[0026] like Figure 1 This is a flow chart of a motion-aware recording method based on monitoring video inter-frame decoding according to the present invention. The motion-aware recording method specifically includes the following steps:

[0027] Step 1: Decode the surveillance video into frames and obtain macroblock information on the frame image.

[0028] Step 2: According to the macroblock information on the adjacent frame images, the macroblock motion vector, macroblock residual, the number of non-zero transform coefficients of the macroblock, the macroblock quantization parameter and the length of the macroblock entropy decoding bit stream are obtained. In a static scene, the macroblock motion vector is close to zero, the macroblock residual is small, the number of non-zero transform coefficients of the macroblock is small, the macroblock quantization parameter is relatively stable, and the length of the macroblock entropy decoding bit stream is also short. In the motion scene of the frame image, the changes of these features are relatively large. Therefore, the present invention obtains the changes of these features on adjacent frames, combines their respective weighting coefficients, and determines the comprehensive motion perception value of the frame image. By analyzing the inter-frame in the surveillance video, the The decoding characteristic starts the recording function only when the frame image motion is detected, improves the accuracy of frame image motion detection, realizes real-time frame image motion detection and recording start, has a fast response speed, avoids the storage of a large number of useless static images, significantly reduces the storage of invalid video clips, saves storage resources, reduces storage costs, supports multiple types of decoding standards, and is suitable for various real-time monitoring scenarios. In addition, the amount of calculation is small, reduces the dependence on high-performance computing resources, reduces energy consumption, and because the calculation of the comprehensive motion perception value of the frame image in the present invention does not involve in-depth analysis of the specific content of the frame image, the privacy of the monitoring video is avoided from being violated. Among them, the comprehensive motion perception value of the frame image includes: the motion vector perception value of the frame image, the residual data perception value of the frame image, the non-zero transform coefficient perception value of the frame image, the quantization perception value of the frame image, and the entropy decoding perception value of the frame image.

[0029] The calculation process of the motion vector perception value of the frame image is as follows: calculate the similarity between each macroblock in the current frame image and the macroblock in the previous frame image, find the macroblock that best matches each macroblock in the current frame image and the previous frame image based on the maximum similarity value, determine the macroblock motion vector of each macroblock in the current frame image relative to the macroblock that best matches the previous frame image, and calculate the standard deviation Vmv of all macroblock motion vectors in the current frame image. If Vmv is large, it means that the motion vector of the current frame image changes greatly and there is a significant displacement change, that is, the current frame image may have obvious motion; otherwise, the current frame image may be in a relatively static state. The standard deviation Vmv of all macroblock motion vectors in the current frame image is combined with the weighting coefficient Wmv of all macroblock motion vectors to obtain the motion vector perception value Pmv=Vmv×Wmv of the current frame image.

[0030] The calculation process of the residual data perception value of the frame image is as follows: according to the maximum value of the similarity, the offset of the pixel point in each macroblock in the current frame image and the corresponding pixel point in the most matching macroblock in the previous frame image is calculated, and the offsets of all the pixel points in each macroblock are superimposed to obtain the macroblock residual. For a static frame image, since the difference between the current frame image and the previous frame image is small, its macroblock residual is also small; on the contrary, for a moving frame image, its macroblock residual is large. Calculate the standard deviation Vrd of all macroblock residuals in the current frame image. The larger Vrd is, the larger the range of variation of the macroblock residual is, and there may be significant pixel changes, that is, the current frame image is likely to be in motion. The standard deviation Vrd of all macroblock residuals in the previous frame image is combined with the weighting coefficient Wrd of all macroblock residuals to obtain the residual data perception value Prd = Vrd × Wrd of the current frame image.

[0031] The calculation process of the non-zero transform coefficient perception value of the frame image is as follows: perform DCT transform on each macroblock in the current frame image, convert the pixel value in each macroblock into a frequency domain coefficient, and count the number of non-zero transform coefficients in each macroblock in the current frame image, which can reflect the distribution of non-zero transform coefficients in the macroblock. Calculate the standard deviation Vcv of the number of non-zero transform coefficients of all macroblocks in the current frame image. The larger the Vcv, the larger the variation range of the non-zero transform coefficients in the macroblock, and there may be significant detail changes, that is, the current frame image is likely to be in motion. The standard deviation Vcv of the number of non-zero transform coefficients of all macroblocks in the current frame image is combined with the weighting coefficient Wcv of the number of non-zero transform coefficients of all macroblocks to obtain the non-zero transform coefficient perception value Pcv=Vcv×Wcv of the current frame image.

[0032] The calculation process of the quantization perception value of the frame image is as follows: according to the maximum similarity value, the change in the quantization parameter of each macroblock in the current frame image and the macroblock quantization parameter of the most matching macroblock in the previous frame image is counted. The change in the macroblock quantization parameter may mean that new detail information has appeared in the current frame image, such as the detail change caused by the movement of people. The standard deviation Vqp of the change in the quantization parameter of all macroblocks in the current frame image is calculated. The larger the Vqp, the larger the change range of the macroblock quantization parameter, which may mean that the image quality or compression strategy has a large change, that is, the current frame image is likely to be in motion. The standard deviation Vqp of the change in the quantization parameter of all macroblocks in the current frame image is combined with the weighting coefficient Wqp of the change in the quantization parameter of all macroblocks to obtain the quantization perception value Pqp = Vqp × Wqp of the current frame image.

[0033] The calculation process of the entropy decoding perception value of the frame image is as follows: according to the maximum similarity value, the difference between the entropy decoding bit stream length of each macroblock in the current frame image and the macroblock entropy decoding bit stream length of the most matching macroblock in the previous frame image is calculated. If the difference in the length of the macroblock entropy decoding bit stream increases significantly, it means that the difference between the current frame image and the previous frame image is large, which may be caused by motion. The standard deviation Vec of the length difference of all macroblock entropy decoding bit streams in the current frame image is calculated. Vec can reflect the distribution of the length of all macroblock entropy decoding bit streams. The larger Vec is, the larger the range of variation of the bit stream length is, and there may be significant content changes, that is, the current frame image is likely to be in motion. The standard deviation Vec of the length difference of all macroblock entropy decoding bit streams in the current frame image is combined with the weighting coefficient Wec of the length difference of all macroblock entropy decoding bit streams to obtain the entropy decoding perception value Pec of the current frame image = Vec × Wec.

[0034] In the present invention, the weighted coefficients Wmv of all macroblock motion vectors of the current frame image, the weighted coefficients Wrd of all macroblock residuals, the weighted coefficients Wcv of the number of non-zero transform coefficients of all macroblocks, the weighted coefficients Wqp of the changes in quantization parameters of all macroblocks, and the weighted coefficients Wec of the length differences of all macroblock entropy decoding bitstreams satisfy:

[0035] Wmv+Wrd+Wcv+Wqp+Wec=1

[0036] Step 3: If the comprehensive motion perception value of the frame image exceeds the set perception threshold, start recording the surveillance video; otherwise, stop recording the surveillance video. The present invention dynamically adjusts the perception threshold to further reduce the false alarm rate, and can perform adaptive optimization based on historical data and real-time feedback to improve the robustness of frame image motion detection. It includes the following sub-steps:

[0037] Step 3.1, collect several consecutive static frame images, obtain the macroblock information on each static frame image, repeat step 2, obtain the comprehensive motion perception value of each static frame image, calculate the average of the comprehensive motion perception values ​​of all static frame images, and determine the initial perception threshold in, represents the mean value of the comprehensive motion perception value of all static frame images, and w represents the perception threshold adjustment coefficient;

[0038] Step 3.2: If the comprehensive motion perception value of the frame image exceeds the initial perception threshold, start recording the surveillance video. If the comprehensive motion perception value of the frame images for one consecutive hour exceeds the perception threshold, use the average of the comprehensive motion perception values ​​of the frame images for one consecutive hour to update the perception threshold. If the comprehensive motion perception value of the frame images for 30 consecutive seconds is lower than the perception threshold, stop recording.

[0039] The motion-aware recording method based on inter-frame decoding of surveillance videos of the present invention utilizes inter-frame decoding technology to analyze the similarity between consecutive frames in surveillance videos and removes redundant information, thereby achieving efficient video compression and supporting various video decoding standards, such as H.264 / AVC, H.265 / HEVC, etc., which can not only effectively reduce the storage of invalid video clips, but also ensure timely recording when important events occur, reduce computing resource requirements, achieve efficient storage, and save storage space.

[0040] In a technical solution of the present invention, an electronic device is also provided, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the motion perception recording method based on inter-frame decoding of surveillance video is implemented.

[0041] In the embodiments disclosed in the present application, the computer storage medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. The computer storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the above. More specific examples of computer storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0042] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0043] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A motion-aware recording method based on inter-frame decoding of surveillance video, characterized in that: The specific steps include: Step 1: Decode the surveillance video into frames and obtain macroblock information on the frame image; Step 2: According to the macroblock information on the adjacent frame images, the macroblock motion vector, the macroblock residual, the number of non-zero transform coefficients of the macroblock, the macroblock quantization parameter and the length of the macroblock entropy decoding bit stream are obtained, and the comprehensive motion perception value of the frame image is determined by combining the respective weighting coefficients; Step 3: If the comprehensive motion perception value of the frame image exceeds the set perception threshold, start recording the surveillance video; otherwise, stop recording the surveillance video.

2. A motion sensing recording method based on inter-frame decoding of surveillance video according to claim 1, characterized in that: The comprehensive motion perception value of the frame image includes: a motion vector perception value of the frame image, a residual data perception value of the frame image, a non-zero transform coefficient perception value of the frame image, a quantization perception value of the frame image, and an entropy decoding perception value of the frame image.

3. The motion-aware recording method based on monitoring video inter-frame decoding according to claim 2, characterized in that: The calculation process of the motion vector perception value of the frame image is as follows: calculate the similarity between each macroblock in the current frame image and the macroblock in the previous frame image, find the macroblock that best matches each macroblock in the current frame image and the previous frame image according to the maximum value of the similarity, determine the macroblock motion vector of each macroblock in the current frame image relative to the macroblock that best matches the previous frame image, calculate the standard deviation Vmv of all macroblock motion vectors in the current frame image, and combine the weighted coefficient Wmv of all macroblock motion vectors to obtain the motion vector perception value Pmv=Vmv×Wmv of the current frame image.

4. The motion-aware recording method based on monitoring video inter-frame decoding according to claim 3 is characterized in that: The calculation process of the residual data perception value of the frame image is as follows: according to the maximum value of the similarity, the offset of the pixel point in each macroblock in the current frame image and the corresponding pixel point in the most matching macroblock in the previous frame image is calculated, the offsets of all pixel points in each macroblock are superimposed to obtain the macroblock residual, the standard deviation Vrd of all macroblock residuals in the current frame image is calculated, and the residual data perception value Prd = Vrd × Wrd of the current frame image is obtained by combining the weighting coefficient Wrd of all macroblock residuals.

5. The motion-aware recording method based on monitoring video inter-frame decoding according to claim 4, characterized in that: The calculation process of the non-zero transform coefficient perception value of the frame image is as follows: perform DCT transform on each macroblock in the current frame image, convert the pixel value in each macroblock into a frequency domain coefficient, count the number of non-zero transform coefficients of each macroblock in the current frame image, calculate the standard deviation Vcv of the number of non-zero transform coefficients of all macroblocks in the current frame image, and combine the weighted coefficient Wcv of the number of non-zero transform coefficients of all macroblocks to obtain the non-zero transform coefficient perception value Pcv=Vcv×Wcv of the current frame image.

6. The motion-aware recording method based on monitoring video inter-frame decoding according to claim 5, characterized in that: The calculation process of the quantization perception value of the frame image is as follows: based on the maximum similarity value, the change in the quantization parameter of each macroblock in the current frame image and the macroblock quantization parameter of the most matching macroblock in the previous frame image is counted, and the standard deviation Vqp of the change in the quantization parameter of all macroblocks in the current frame image is calculated. Combined with the weighted coefficient Wqp of the change in the quantization parameter of all macroblocks, the quantization perception value Pqp of the current frame image is obtained as Pqp=Vqp×Wqp.

7. The motion-aware recording method based on monitoring video inter-frame decoding according to claim 6, characterized in that: The calculation process of the entropy decoding perception value of the frame image is as follows: according to the maximum similarity value, the difference between the entropy decoding bit stream length of each macroblock in the current frame image and the macroblock entropy decoding bit stream length of the most matching macroblock in the previous frame image is calculated, and the standard deviation Vec of the entropy decoding bit stream length difference of all macroblocks in the current frame image is calculated. Combined with the weighted coefficient Wec of the entropy decoding bit stream length difference of all macroblocks, the entropy decoding perception value Pec=Vec×Wec of the current frame image is obtained.

8. The motion-aware recording method based on monitoring video inter-frame decoding according to claim 7, characterized in that: Wmv+Wrd+Wcv+Wqp+Wec=1.

9. The motion-aware recording method based on monitoring video inter-frame decoding according to claim 8, characterized in that: Step 3 includes the following sub-steps: Step 3.1, collect several consecutive static frame images, obtain the macroblock information on each static frame image, repeat step 2, obtain the comprehensive motion perception value of each static frame image, calculate the average of the comprehensive motion perception values ​​of all static frame images, and determine the initial perception threshold in, represents the mean value of the comprehensive motion perception value of all static frame images, and w represents the perception threshold adjustment coefficient; Step 3.2: If the comprehensive motion perception value of the frame image exceeds the initial perception threshold, start recording the surveillance video. If the comprehensive motion perception value of the frame images for one consecutive hour exceeds the perception threshold, use the average of the comprehensive motion perception values ​​of the frame images for one consecutive hour to update the perception threshold. If the comprehensive motion perception value of the frame images for 30 consecutive seconds is lower than the perception threshold, stop recording.

10. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the motion perception recording method based on inter-frame decoding of surveillance video as described in any one of claims 1 to 9 is implemented.

Images