A moving target detection method, device and terminal equipment

Abstract

The present invention discloses a moving target detection method, device and terminal equipment, including: obtaining an image to be detected, detecting a moving foreground area of ​​the image to be detected by a motion detection method; performing macroblock division processing according to the image to be detected and the moving foreground area to obtain a macroblock foreground image; calculating the mutual information between adjacent macroblocks according to the macroblock foreground image to obtain a mutual information correlation image; performing region growth processing on the macroblock foreground image according to the mutual information correlation image to obtain a processed macroblock foreground image; obtaining the historical time sequence information of each macroblock in the processed macroblock foreground image, combining with a Bayesian filter, judging whether each macroblock in the processed macroblock foreground image is a foreground; and obtaining a moving target area according to all macroblocks judged to be foregrounds. The technical solution of the present invention can have good robustness to noise, enhance anti-interference performance, and thus improve the accuracy of moving target detection.

Description

Technical Field

[0001] The present invention relates to the field of monitoring and image processing technology, and in particular to a moving target detection method, device and terminal equipment. Background Art

[0002] At present, the mainstream methods for detecting moving targets mainly include optical flow method and background difference method. Among them, the general steps of the optical flow method are to determine the grayscale changes and the correlation between adjacent pixels at different times through the changes in pixel speed in the image sequence, so as to detect the moving target. The background difference method is to first construct a background model to replace the real background scene, and then compare the image sequence with the background model to identify the difference between the moving target and the background to achieve the detection of the moving target. Typical background models include mixed Gaussian model, ViBe, etc.

[0003] However, the optical flow method is susceptible to noise and has poor anti-interference performance. The background difference method is easily affected by dynamic changes in the background scene (such as swaying leaves, ripples on the lake surface, weather changes), light changes, and cluttered backgrounds. In addition, for slowly moving targets, their pixels can easily be integrated into the background model too quickly, resulting in low accuracy in moving target detection and the possibility of misidentifying dynamic backgrounds as moving targets. Summary of the invention

[0004] The technical problem to be solved by the embodiments of the present invention is to provide a moving target detection method, apparatus and terminal device, which can have good robustness against noise and enhance anti-interference performance, thereby improving the accuracy of moving target detection.

[0005] In order to solve the above technical problems, an embodiment of the present invention provides a moving target detection method, comprising:

[0006] Acquire an image to be detected, and detect a motion foreground area of ​​the image to be detected by a motion detection method;

[0007] Performing macroblock division processing according to the image to be detected and the motion foreground area to obtain a macroblock foreground image;

[0008] Calculate mutual information between adjacent macroblocks based on the macroblock foreground image to obtain a mutual information correlation image;

[0009] Performing region growing processing on the macroblock foreground image according to the mutual information correlation image to obtain a processed macroblock foreground image;

[0010] Acquire historical time sequence information of each macroblock in the processed macroblock foreground image, and determine whether each macroblock in the processed macroblock foreground image is a foreground by combining with a Bayesian filter;

[0011] The moving object area is obtained based on all macroblocks determined to be foreground.

[0012] Furthermore, the acquiring of the image to be detected and detecting the motion foreground area of ​​the image to be detected by a motion detection method specifically includes:

[0013] Acquire a sequence of images to be processed; wherein the sequence of images to be processed includes the image to be detected;

[0014] The motion detection method is used to detect several frames of images in the image sequence to be processed to obtain the motion foreground area.

[0015] Furthermore, the method further comprises:

[0016] Performing morphological processing on the motion foreground region to obtain a processed motion foreground region;

[0017] Then, the macroblock division process is performed according to the image to be detected and the motion foreground area to obtain a macroblock foreground image, specifically including:

[0018] A macroblock division process is performed according to the image to be detected and the processed motion foreground area to obtain a macroblock foreground image.

[0019] Further, the calculating the mutual information between adjacent macroblocks according to the macroblock foreground image to obtain the mutual information correlation image specifically includes:

[0020] For any two adjacent macroblocks a and b in the macroblock foreground image, the mutual information I(a,b) between macroblock a and macroblock b is calculated according to the formula I(a,b)=H(a)+H(b)-H(a,b); wherein H(a) represents the entropy of macroblock a, H(b) represents the entropy of macroblock b, and H(a,b) represents the joint entropy of macroblock a and macroblock b;

[0021] The mutual information correlation image is obtained according to the mutual information between all two adjacent macroblocks in the macroblock foreground image.

[0022] Furthermore, the method further comprises:

[0023] Processing the mutual information correlation image by using a Bayesian filter to obtain a processed mutual information correlation image;

[0024] Then, performing region growing processing on the macroblock foreground image according to the mutual information correlation image to obtain the processed macroblock foreground image specifically includes:

[0025] A region growing process is performed on the macroblock foreground image according to the processed mutual information correlation image to obtain a processed macroblock foreground image.

[0026] Further, performing region growing processing on the macroblock foreground image according to the mutual information correlation image to obtain the processed macroblock foreground image specifically includes:

[0027] For each macroblock in the macroblock foreground image, when the mutual information between the macroblock and any adjacent macroblock is greater than a preset mutual information threshold, marking the adjacent macroblock as a connected domain of the macroblock;

[0028] Performing mean filtering on the mutual information in each connected domain of the macroblock, and when the mean filtering value corresponding to any connected domain is greater than a preset mean threshold, performing region growing processing on the macroblock in a direction corresponding to the connected domain;

[0029] The processed macroblock foreground image is obtained according to all macroblocks in the macroblock foreground image and all macroblocks processed by region growing.

[0030] Further, the acquiring of the historical timing information of each macroblock in the processed macroblock foreground image and combining with the Bayesian filter to determine whether each macroblock in the processed macroblock foreground image is a foreground specifically includes:

[0031] Obtaining the historical timing information of each macroblock in the processed macroblock foreground image; wherein the historical timing information of any macroblock in the processed macroblock foreground image is expressed as [ST i (1),ST i (2),...,ST i (t-1),ST i (t)],ST i (t) represents the probability that the macroblock is determined as the foreground by the motion detection method at time t in the i-th frame image, i>0;

[0032] Combined with the Bayesian filter, the probability of each macroblock in the processed macroblock foreground image being determined as the foreground is calculated; wherein the probability of any macroblock in the processed macroblock foreground image being determined as the foreground is expressed as η represents the normalization coefficient, η>0, q represents the preset time range, q>0, P(ST i (t)|fg) represents the conditional probability, and P(fg) represents the probability that the macroblock belongs to the foreground;

[0033] When any macroblock in the processed macroblock foreground image is determined to be the foreground, the probability ST i When (t) is greater than a preset probability threshold, the macroblock is determined to be a foreground.

[0034] Furthermore, the method further comprises:

[0035] According to the probability that each macroblock in the processed macroblock foreground image is determined to be the foreground, the probability parameter corresponding to each macroblock is updated accordingly; wherein, for any macroblock in the processed macroblock foreground image, according to ST i '(t) for the conditional probability P(ST i (t)|fg) and probability P(fg) are updated accordingly.

[0036] In order to solve the above technical problems, an embodiment of the present invention further provides a moving target detection device, comprising:

[0037] A motion foreground area acquisition module is used to acquire an image to be detected and detect the motion foreground area of ​​the image to be detected by a motion detection method;

[0038] A macroblock foreground image acquisition module, used for performing macroblock division processing according to the image to be detected and the motion foreground area to obtain a macroblock foreground image;

[0039] A mutual information related image acquisition module, used for calculating the mutual information between adjacent macroblocks according to the macroblock foreground image, and obtaining a mutual information related image;

[0040] A macroblock foreground image processing module, used for performing region growing processing on the macroblock foreground image according to the mutual information correlation image to obtain a processed macroblock foreground image;

[0041] A foreground macroblock judgment module is used to obtain the historical timing information of each macroblock in the processed macroblock foreground image, and determine whether each macroblock in the processed macroblock foreground image is a foreground by combining with a Bayesian filter;

[0042] The moving target area acquisition module is used to obtain the moving target area according to all macroblocks determined as foreground.

[0043] An embodiment of the present invention also provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements any of the above-mentioned moving target detection methods when executing the computer program.

[0044] Compared with the prior art, the embodiments of the present invention provide a method, apparatus and terminal device for detecting a moving target. First, an image to be detected is obtained, a moving foreground area of ​​the image to be detected is detected by a motion detection method, and macroblock division processing is performed according to the image to be detected and the moving foreground area to obtain a macroblock foreground image. Then, mutual information between adjacent macroblocks is calculated according to the macroblock foreground image to obtain a mutual information correlation image. Region growth processing is performed on the macroblock foreground image according to the mutual information correlation image to obtain a processed macroblock foreground image, and historical timing information of each macroblock in the processed macroblock foreground image is obtained. Combined with a Bayesian filter, it is determined whether each macroblock in the processed macroblock foreground image is a foreground. Finally, a moving target area is obtained according to all macroblocks determined to be foregrounds. The embodiments of the present invention can have good robustness to noise and enhance anti-interference performance, thereby improving the accuracy of moving target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flow chart of a preferred embodiment of a moving target detection method provided by the present invention;

[0046] Figure 2 It is a structural block diagram of a preferred embodiment of a moving target detection device provided by the present invention;

[0047] Figure 3 It is a structural block diagram of a preferred embodiment of a terminal device provided by the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this technical field without creative work are within the scope of protection of the present invention.

[0049] The present invention provides a moving target detection method. Figure 1 FIG. 1 is a flow chart of a preferred embodiment of a moving target detection method provided by the present invention, wherein the method comprises steps S11 to S16:

[0050] Step S11, obtaining an image to be detected, and detecting a motion foreground area of ​​the image to be detected by a motion detection method;

[0051] Step S12, performing macroblock division processing according to the image to be detected and the motion foreground area to obtain a macroblock foreground image;

[0052] Step S13, calculating the mutual information between adjacent macroblocks according to the macroblock foreground image to obtain a mutual information correlation image;

[0053] Step S14, performing region growing processing on the macroblock foreground image according to the mutual information correlation image to obtain a processed macroblock foreground image;

[0054] Step S15, obtaining the historical timing information of each macroblock in the processed macroblock foreground image, and combining with the Bayesian filter to determine whether each macroblock in the processed macroblock foreground image is a foreground;

[0055] Step S16: Obtain a moving target area based on all macroblocks determined to be foreground.

[0056] Specifically, the image to be detected can be acquired in real time through an electronic device. For example, the image to be detected can be acquired in real time through a video data stream recorded by an electronic device with a video recording function such as a webcam, a mobile phone, or a tablet computer. The image to be detected can also be acquired by performing non-real-time processing based on the recorded video. The embodiment of the present invention does not specifically limit the acquisition method.

[0057] After obtaining the image to be detected, the existing motion detection method is used to perform corresponding processing on the image to be detected to detect the motion foreground area in the image to be detected. For example, the GMM Gaussian mixture model of the scene can be used to obtain the motion foreground area in the image to be detected, or ViBe, K nearest neighbor algorithm, optical flow method, and methods based on deep learning, neural networks, and methods based on image segmentation can be used to obtain the motion foreground area in the image to be detected. The motion detection method is not limited to the color space, and can be grayscale, RGB, YUV, HSV, etc., which is not specifically limited in the embodiment of the present invention.

[0058] After obtaining the motion foreground area in the image to be detected, first, the image to be detected and the motion foreground area in the image to be detected are divided into macroblocks, and a macroblock foreground image is obtained accordingly. For example, each m×n pixel can be divided into a macroblock, m≥1, n≥1, and the macroblocks can be arranged adjacently or with aliasing, and the macroblock representation includes but is not limited to a circumscribed rectangle, a circumscribed ellipse, a circumscribed rhombus and its equivalent data representation, etc.; then, the mutual information between all adjacent macroblocks (adjacent macroblocks include but are not limited to adjacent four-connected domains, adjacent eight-connected domains, etc.) is calculated according to the obtained macroblock foreground image, and a mutual information correlation image is obtained accordingly, and the macroblock foreground image is subjected to region growing processing according to the obtained mutual information correlation image to obtain a processed macroblock foreground image; then, the historical time sequence information of each macroblock in the processed macroblock foreground image is obtained and recorded, and according to the historical time sequence information of each macroblock and the Bayesian filter, each macroblock in the processed macroblock foreground image is jointly judged whether it belongs to the foreground, and the judgment results corresponding to all macroblocks in the processed macroblock foreground image are obtained accordingly, so as to obtain the motion target area according to all macroblocks judged to belong to the foreground.

[0059] A method for detecting a moving target provided by an embodiment of the present invention comprises the following steps: first, obtaining an image to be detected, detecting a moving foreground area of ​​the image to be detected by a motion detection method, performing macroblock division processing based on the image to be detected and the moving foreground area to obtain a macroblock foreground image, then calculating mutual information between adjacent macroblocks based on the macroblock foreground image to obtain a mutual information correlation image, performing region growth processing on the macroblock foreground image based on the mutual information correlation image to obtain a processed macroblock foreground image, and obtaining historical timing information of each macroblock in the processed macroblock foreground image, combining with a Bayesian filter to determine whether each macroblock in the processed macroblock foreground image is a foreground, and finally obtaining a moving target area based on all macroblocks determined to be foregrounds; the embodiment of the present invention can have good robustness to noise, enhance anti-interference performance, and thus improve the accuracy of moving target detection.

[0060] In another preferred embodiment, the acquiring of the image to be detected and detecting the motion foreground area of ​​the image to be detected by a motion detection method specifically includes:

[0061] Acquire a sequence of images to be processed; wherein the sequence of images to be processed includes the image to be detected;

[0062] The motion detection method is used to detect several frames of images in the image sequence to be processed to obtain the motion foreground area.

[0063] Specifically, in combination with the above embodiments, when extracting the motion foreground area in the image to be detected, a sequence of images to be processed (or video, equivalent two-dimensional signal sequence, etc.) can be obtained first, and the sequence of images to be processed includes the image to be detected, and then the existing motion detection method is used to detect several frames of images in the sequence of images to be processed to detect the motion foreground area in the image to be detected.

[0064] It should be noted that the embodiments of the present invention can detect each frame image in the image sequence to be processed to obtain the corresponding motion foreground area. However, on some embedded devices, due to insufficient performance or resources, frame skipping processing may be performed on the image sequence to be processed. For example, a number of frames of images in the image sequence to be processed are selected for detection every two frames, or the image sequence to be processed is processed using a set dynamic frame rate. The specific number of frame skipping can be evaluated based on the video information (for example, the degree of historical picture changes), and the embodiments of the present invention do not make specific limitations.

[0065] It is understandable that when there is no frame skipping, the corresponding resource consumption is higher (such as using more CPU), but the timeliness and effect of processing are often better. Frame skipping can control resource consumption to a certain extent. In actual projects, a compromise evaluation is often performed.

[0066] In another preferred embodiment, the method further comprises:

[0067] Performing morphological processing on the motion foreground region to obtain a processed motion foreground region;

[0068] Then, the macroblock division process is performed according to the image to be detected and the motion foreground area to obtain a macroblock foreground image, specifically including:

[0069] A macroblock division process is performed according to the image to be detected and the processed motion foreground area to obtain a macroblock foreground image.

[0070] Specifically, in combination with the above embodiments, after obtaining the motion foreground area in the image to be detected, in order to make the obtained motion foreground area more accurate, a morphological processing operation can be performed on the obtained motion foreground area to obtain a corrected motion foreground area, and accordingly, a macroblock division operation is performed on the image to be detected and the corrected motion foreground area to obtain a macroblock foreground image accordingly.

[0071] It should be noted that performing morphological operations on the obtained motion foreground area can reduce noise and smooth the boundaries of the motion foreground area to achieve better results. Morphological operations include but are not limited to opening operations, closing operations, corrosion, expansion, top-hat changes, black-hat operations, etc. However, for some specific scenes, there may be situations where no morphological operations can still achieve good results. Therefore, in this case, the morphological operations also include no operations.

[0072] In another preferred embodiment, the step of calculating the mutual information between adjacent macroblocks based on the macroblock foreground image to obtain the mutual information correlation image specifically includes:

[0073] For any two adjacent macroblocks a and b in the macroblock foreground image, the mutual information I(a,b) between macroblock a and macroblock b is calculated according to the formula I(a,b)=H(a)+H(b)-H(a,b); wherein H(a) represents the entropy of macroblock a, H(b) represents the entropy of macroblock b, and H(a,b) represents the joint entropy of macroblock a and macroblock b;

[0074] The mutual information correlation image is obtained according to the mutual information between all two adjacent macroblocks in the macroblock foreground image.

[0075] Specifically, in combination with the above embodiment, when calculating the mutual information between adjacent macroblocks, taking any two adjacent macroblocks a and macroblock b in the macroblock foreground image as an example, the mutual information I(a,b) between macroblock a and macroblock b can be calculated according to the following formula: I(a,b)=H(a)-H(a|b)=H(a)+H(b)-H(a,b), where H(a) represents the entropy of macroblock a, H(b) represents the entropy of macroblock b, and H(a,b) represents the joint entropy of macroblock a and macroblock b; similarly, the same calculation method can be used to obtain the mutual information between all two adjacent macroblocks in the macroblock foreground image, and based on the obtained mutual information between all two adjacent macroblocks, a mutual information related image can be obtained accordingly.

[0076] Among them, for the entropy of the macroblock, taking the entropy of macroblock a as an example, the entropy of macroblock a can be calculated according to the following formula: i is the number of iterations, which means for i = 1 to N, where N is the capacity of historical point statistics, such as recording the first N frames of this macroblock. Generally, the larger N is, the more resources it consumes, but it is relatively more accurate. i Represents the value of macroblock a recorded in the i-th frame. p(a i ) represents the statistical probability that macroblock a takes the value of the i-th frame.

[0077] For example, after a macroblock is synthesized by 4 pixels, if there is no aliasing, then in the macroblock foreground image, the neighborhood on the right side of the macroblock is a macroblock synthesized by the 4 pixels on the right. Correspondingly, the above method can be used to calculate the mutual information between the two macroblocks. After obtaining the mutual information between all two adjacent macroblocks in the macroblock foreground image, they are adjacent to each other to form a two-dimensional graph, and the mutual information related image can be obtained accordingly.

[0078] It is understandable that if the background of the image is snowing, raining or leaves are shaking, and there is a person in the middle of the image, the pixels of the person are related to each other (in other words, there is a head and eyes, and there is a nose next to them). Therefore, the mutual information of the person's moving foreground area is relatively high, while the leaves in the background are shaking all the time, and the mutual information is relatively low. The final effect of the mutual information correlation image is like the entire related object (such as a person) is marked out.

[0079] In another preferred embodiment, the method further comprises:

[0080] Processing the mutual information correlation image through a Bayesian filter to obtain a processed mutual information correlation image;

[0081] Then, performing region growing processing on the macroblock foreground image according to the mutual information correlation image to obtain the processed macroblock foreground image specifically includes:

[0082] A region growing process is performed on the macroblock foreground image according to the processed mutual information correlation image to obtain a processed macroblock foreground image.

[0083] Specifically, in combination with the above embodiments, after obtaining the mutual information correlation image, the obtained mutual information correlation image can be subjected to Bayesian filtering processing through a Bayesian filter to obtain a stable mutual information correlation image under time domain conditions, that is, to obtain the mutual information correlation image after filtering processing. Correspondingly, the macroblock foreground image is subjected to regional growing processing based on the obtained mutual information correlation image after filtering processing to obtain the processed macroblock foreground image.

[0084] It should be noted that, for adjacent macroblocks in the macroblock foreground image, while calculating the mutual information between adjacent macroblocks, the mutual information history values ​​of each adjacent macroblock can be recorded, so as to filter the mutual information correlation image based on the mutual information history values ​​of the adjacent macroblocks and then use the Bayesian filter to filter the mutual information correlation image, thereby filtering out the noise in the time domain in the mutual information correlation image and generating a stable, filtered mutual information correlation image accordingly.

[0085] In another preferred embodiment, performing region growing processing on the macroblock foreground image according to the mutual information correlation image to obtain the processed macroblock foreground image specifically includes:

[0086] For each macroblock in the macroblock foreground image, when the mutual information between the macroblock and any adjacent macroblock is greater than a preset mutual information threshold, marking the adjacent macroblock as a connected domain of the macroblock;

[0087] Performing mean filtering on the mutual information in each connected domain of the macroblock, and when the mean filtering value corresponding to any connected domain is greater than a preset mean threshold, performing region growing processing on the macroblock in a direction corresponding to the connected domain;

[0088] The processed macroblock foreground image is obtained according to all macroblocks in the macroblock foreground image and all macroblocks processed by region growing.

[0089] Specifically, in combination with the above-mentioned embodiment, when performing region growing processing on the macroblock foreground image according to the obtained mutual information correlation image (or the mutual information correlation image after filtering), a typical region growing method provided by the prior art can be adopted, that is, performing mean filtering on the mutual information in the connected domain (such as a four-connected domain, etc.) of a certain macroblock, and judging whether it is greater than a certain threshold value, if it is greater than, it is considered to be relevant, and region growing is performed in this direction. In specific implementation, taking any macroblock in the macroblock foreground image as an example, when the mutual information between the macroblock and any of its adjacent macroblocks is greater than a preset value, the region growing method is performed. When the mutual information threshold is reached, the adjacent macroblock is marked as the connected domain of the macroblock, all the connected domains of the macroblock are obtained accordingly, and the mutual information in each connected domain of the macroblock is subjected to mean filtering. When the mean filtering value corresponding to any connected domain of the macroblock is greater than the preset mean threshold, the macroblock is subjected to region growing processing in the direction corresponding to the connected domain. Similarly, the same processing method can be used to perform region growing processing on each macroblock in the macroblock foreground image, so as to obtain the processed macroblock foreground image according to all the macroblocks in the macroblock foreground image and all the macroblocks after region growing processing.

[0090] For example, for each macroblock, determine whether the four mutual information adjacent to it meet the threshold condition. If so, it is marked as a connected domain. The connected domain is directional. If the mutual information corresponding to the direction of the four directions meets the threshold condition, a mark is generated in that direction as the same connected domain. The points in the same connected domain are mean filtered. If the value obtained by mean filtering is greater than the mean threshold, it is determined to be the foreground.

[0091] In another preferred embodiment, the acquiring of the historical timing information of each macroblock in the processed macroblock foreground image and combining with a Bayesian filter to determine whether each macroblock in the processed macroblock foreground image is a foreground specifically includes:

[0092] Obtaining the historical timing information of each macroblock in the processed macroblock foreground image; wherein the historical timing information of any macroblock in the processed macroblock foreground image is expressed as [ST i (1),ST i (2),...,ST i (t-1),ST i (t)],ST i (t) represents the probability that the macroblock is determined as the foreground by the motion detection method at time t in the i-th frame image, i>0;

[0093] Combined with the Bayesian filter, the probability of each macroblock in the processed macroblock foreground image being determined as the foreground is calculated; wherein the probability of any macroblock in the processed macroblock foreground image being determined as the foreground is expressed as η represents the normalization coefficient, η>0, q represents the preset time range, q>0, P(ST i (t)|fg) represents the conditional probability, and P(fg) represents the probability that the macroblock belongs to the foreground;

[0094] When any macroblock in the processed macroblock foreground image is determined to be the foreground, the probability ST i When (t) is greater than a preset probability threshold, the macroblock is determined to be a foreground.

[0095] Specifically, in combination with the above embodiment, after obtaining the processed macroblock foreground image, the historical timing information corresponding to each macroblock in the processed macroblock foreground image is first obtained and recorded. The historical timing information of any macroblock in the processed macroblock foreground image is expressed as [ST i (1),ST i (2),...,ST i (t-1),ST i (t)],ST i (t) represents the probability that the macroblock is determined as the foreground by the existing motion detection method at time t in the i-th frame image, i>0; then, according to the historical time sequence information corresponding to all macroblocks and using the Bayesian filter, the probability that each macroblock in the processed macroblock foreground image is determined as the foreground is obtained by joint calculation. The probability that any macroblock in the processed macroblock foreground image is determined as the foreground is expressed as η represents the normalization coefficient, η>0, q represents the preset time range, q>0, P(ST i (t)|fg) represents the conditional probability, P(fg) represents the probability that the macroblock belongs to the foreground; accordingly, the probability ST iWhen '(t) is greater than a preset probability threshold (the probability threshold can be adjusted according to the sensitivity, or the user can slide the slider on the user interface to adjust the sensitivity), the macroblock is determined to be the foreground.

[0096] Among them, for the historical timing information corresponding to each macroblock, the historical timing information may include complete historical timing information or truncated historical timing information, for example, only retaining the historical timing information of the most recent few frames, or the historical timing information of several frames in the case of frame skipping / dynamic frame skipping.

[0097] When acquiring and recording the historical timing information corresponding to each macroblock in the processed macroblock foreground image, it is usually necessary to perform corresponding operations on each macroblock. However, macroblocks can also be selected based on the performance of the device. For example, only the historical timing information corresponding to macroblocks whose area size is greater than a certain threshold is counted and recorded, thereby reducing the amount of data counted and recorded.

[0098] It should be noted that the foreground result of a macroblock in the macroblock foreground image obtained by the existing motion detection method is marked as ST i (t), the foreground result of the current frame correction of this macroblock is ST i '(t), the Bayesian filter formula is: In short: The normalization coefficient is: The final formula is: P(fg|s 1 ,s 2 ,...,s n )=η·∏P(s i |fg)P(fg).

[0099] As an improvement of the above solution, the method further includes:

[0100] According to the probability that each macroblock in the processed macroblock foreground image is determined to be the foreground, the probability parameter corresponding to each macroblock is updated accordingly; wherein, for any macroblock in the processed macroblock foreground image, according to ST i '(t) for the conditional probability P(ST i (t)|fg) and probability P(fg) are updated accordingly.

[0101] Specifically, in combination with the above embodiment, after obtaining the probability that each macroblock in the processed macroblock foreground image is determined to be the foreground, the probability parameter corresponding to each macroblock can be updated accordingly according to the probability that each macroblock in the processed macroblock foreground image is determined to be the foreground. Taking any macroblock in the processed macroblock foreground image as an example, the probability parameter corresponding to each macroblock can be updated according to the ST i'(t) for the conditional probability P(ST i (t)|fg) and probability P(fg) are updated accordingly.

[0102] It should be noted that, normally each frame of the image is updated accordingly, however, in specific engineering optimization situations, dynamic updates can also be performed as needed, for example, frame skipping updates, or updating after the amplitude is greater than a specific threshold, which is not specifically limited in the embodiments of the present invention.

[0103] It is understandable that the purpose of updating parameters is to make a better foreground probability estimate next time, and to calculate the prior probability based on the posterior probability. For example, a point is the foreground because of the shaking of leaves, or because of the light, or because of the noise of the equipment. First, combine historical information to predict and judge whether it is the foreground, and the basis for judgment is the previously statistical probability. Then, after this judgment is completed, it is necessary to use this record to update the previously statistical probability. Assuming that it is determined to be the foreground due to light, then it is necessary to add 1 to the event count of the light being judged as the foreground. The possibility of this point being judged as the foreground due to light increases. In subsequent judgments, it is more inclined to think that this is a foreground point caused by light. By updating parameters, the judgment error can be reduced.

[0104] The present invention also provides a moving target detection device. Figure 2 FIG. 1 is a structural block diagram of a preferred embodiment of a moving target detection device provided by the present invention, wherein the device comprises:

[0105] The motion foreground region acquisition module 11 is used to acquire the image to be detected and detect the motion foreground region of the image to be detected by a motion detection method;

[0106] A macroblock foreground image acquisition module 12 is used to perform macroblock division processing according to the image to be detected and the motion foreground area to obtain a macroblock foreground image;

[0107] A mutual information related image acquisition module 13 is used to calculate the mutual information between adjacent macroblocks according to the macroblock foreground image to obtain a mutual information related image;

[0108] A macroblock foreground image processing module 14, configured to perform region growing processing on the macroblock foreground image according to the mutual information correlation image to obtain a processed macroblock foreground image;

[0109] A foreground macroblock determination module 15 is used to obtain the historical timing information of each macroblock in the processed macroblock foreground image, and determine whether each macroblock in the processed macroblock foreground image is a foreground by combining with a Bayesian filter;

[0110] The moving object region acquisition module 16 is used to obtain the moving object region according to all macroblocks determined as foreground.

[0111] Preferably, the motion foreground area acquisition module 11 specifically includes:

[0112] An image sequence acquisition unit, used for acquiring an image sequence to be processed; wherein the image sequence to be processed includes the image to be detected;

[0113] The motion foreground area acquisition unit is used to detect a plurality of frame images in the image sequence to be processed by the motion detection method to obtain the motion foreground area.

[0114] Preferably, the device further comprises:

[0115] A motion foreground region processing module, used for performing morphological processing on the motion foreground region to obtain a processed motion foreground region;

[0116] Then, the macroblock foreground image acquisition module 12 is specifically used for:

[0117] A macroblock division process is performed according to the image to be detected and the processed motion foreground area to obtain a macroblock foreground image.

[0118] Preferably, the mutual information related image acquisition module 13 specifically includes:

[0119] A mutual information calculation unit, for calculating the mutual information I(a,b) between macroblock a and macroblock b according to the formula I(a,b)=H(a)+H(b)-H(a,b) for any two adjacent macroblocks a and macroblock b in the macroblock foreground image; wherein H(a) represents the entropy of macroblock a, H(b) represents the entropy of macroblock b, and H(a,b) represents the joint entropy of macroblock a and macroblock b;

[0120] The mutual information related image acquisition unit is used to obtain the mutual information related image according to the mutual information between all two adjacent macroblocks in the macroblock foreground image.

[0121] Preferably, the method further comprises:

[0122] A mutual information correlation image processing module, used for processing the mutual information correlation image through a Bayesian filter to obtain a processed mutual information correlation image;

[0123] Then, the macroblock foreground image processing module 14 is specifically used for:

[0124] A region growing process is performed on the macroblock foreground image according to the processed mutual information correlation image to obtain a processed macroblock foreground image.

[0125] Preferably, the macroblock foreground image processing module 14 specifically includes:

[0126] A connected domain marking unit, for each macroblock in the macroblock foreground image, when the mutual information between the macroblock and any adjacent macroblock is greater than a preset mutual information threshold, marking the adjacent macroblock as a connected domain of the macroblock;

[0127] A region growing processing unit, configured to perform mean filtering on the mutual information in each connected domain of the macroblock, and when the mean filtering value corresponding to any connected domain is greater than a preset mean threshold, perform region growing processing on the macroblock in a direction corresponding to the connected domain;

[0128] The macroblock foreground image processing unit is used to obtain the processed macroblock foreground image according to all macroblocks in the macroblock foreground image and all macroblocks processed by region growth.

[0129] Preferably, the foreground macroblock determination module 15 specifically includes:

[0130] A historical timing information acquisition unit is used to acquire the historical timing information of each macroblock in the processed macroblock foreground image; wherein the historical timing information of any macroblock in the processed macroblock foreground image is expressed as [ST i (1),ST i (2),...,ST i (t-1),ST i (t)],ST i (t) represents the probability that the macroblock is determined as the foreground by the motion detection method at time t in the i-th frame image, i>0;

[0131] The foreground probability calculation unit is used to calculate the probability of each macroblock in the processed macroblock foreground image being determined as the foreground in combination with the Bayesian filter; wherein the probability of any macroblock in the processed macroblock foreground image being determined as the foreground is expressed as η represents the normalization coefficient, η>0, q represents the preset time range, q>0, P(ST i (t)|fg) represents the conditional probability, and P(fg) represents the probability that the macroblock belongs to the foreground;

[0132] A foreground macroblock judging unit is used to determine the probability ST when any macroblock in the processed macroblock foreground image is judged to be the foreground. i When (t) is greater than a preset probability threshold, the macroblock is determined to be a foreground.

[0133] Preferably, the device further comprises:

[0134] A probability parameter updating module is used to update the probability parameter corresponding to each macroblock according to the probability that each macroblock in the processed macroblock foreground image is determined to be the foreground; wherein, for any macroblock in the processed macroblock foreground image, according to ST i '(t) for the conditional probability P(ST i (t)|fg) and probability P(fg) are updated accordingly.

[0135] It should be noted that a motion target detection device provided by an embodiment of the present invention can implement all the processes of the motion target detection method described in any of the above embodiments. The functions of each module and unit in the device and the technical effects achieved are respectively the same as the functions and technical effects achieved by the motion target detection method described in the above embodiments, and will not be repeated here.

[0136] The embodiment of the present invention also provides a terminal device, see Figure 3 As shown, it is a structural block diagram of a preferred embodiment of a terminal device provided by the present invention, wherein the terminal device includes a processor 10, a memory 20, and a computer program stored in the memory 20 and configured to be executed by the processor 10, and the processor 10 implements the motion target detection method described in any of the above embodiments when executing the computer program.

[0137] Preferably, the computer program can be divided into one or more modules / units (such as computer program 1, computer program 2, ...), and the one or more modules / units are stored in the memory 20 and executed by the processor 10 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program in the terminal device.

[0138] The processor 10 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor 10 may be any conventional processor. The processor 10 is the control center of the terminal device, and various parts of the terminal device are connected using various interfaces and lines.

[0139] The memory 20 mainly includes a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function, etc., and the data storage area can store related data, etc. In addition, the memory 20 can be a high-speed random access memory, or a non-volatile memory, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, and a flash card (Flash Card), etc., or the memory 20 can also be other volatile solid-state storage devices.

[0140] It should be noted that the above terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that Figure 3 The structural block diagram is merely an example of the above-mentioned terminal device and does not constitute a limitation on the terminal device. The terminal device may include more or less components than shown in the figure, or a combination of certain components, or different components.

[0141] In summary, the moving target detection method, device and terminal device provided by the embodiments of the present invention have the following beneficial effects:

[0142] (1) Since real moving targets (such as people) are continuous in space, mutual information and region growing can effectively filter out the influence of local noise on the detection results, thereby enhancing the anti-interference performance and having good robustness to noise, thereby improving the accuracy of moving target detection;

[0143] (2) By combining historical statistical data, the influence of noise can be effectively suppressed, making the detection results more stable and accurate, and instantaneous jitter will not affect the judgment results;

[0144] (3) By using macroblocks, spatial relationships and Bayesian theory to adaptively learn the detection process, adaptive and anti-interference motion detection is achieved on devices with limited computing power, such as embedded devices. The method has strong adaptability and statistical probability calculation, which can achieve good detection results for multiple types of motion detection scenarios.

[0145] (4) Compared with existing deep learning-based time series models (such as LSTM), it has the advantages of small computational complexity and fast speed.

[0146] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A moving target detection method, characterized in that: include: Acquire an image to be detected, and detect a motion foreground area of ​​the image to be detected by a motion detection method; Performing macroblock division processing according to the image to be detected and the motion foreground area to obtain a macroblock foreground image; Calculate mutual information between adjacent macroblocks based on the macroblock foreground image to obtain a mutual information correlation image; Performing region growing processing on the macroblock foreground image according to the mutual information correlation image to obtain a processed macroblock foreground image; Acquire historical time sequence information of each macroblock in the processed macroblock foreground image, and determine whether each macroblock in the processed macroblock foreground image is a foreground by combining with a Bayesian filter; Obtaining a moving target area based on all macroblocks determined as foregrounds; The step of calculating the mutual information between adjacent macroblocks according to the macroblock foreground image to obtain the mutual information related image specifically includes: For any two adjacent macroblocks a and b in the macroblock foreground image, the mutual information I(a,b) between macroblock a and macroblock b is calculated according to the formula I(a,b)=H(a)+H(b)-H(a,b); wherein H(a) represents the entropy of macroblock a, H(b) represents the entropy of macroblock b, and H(a,b) represents the joint entropy of macroblock a and macroblock b; Obtain the mutual information correlation image according to the mutual information between all two adjacent macroblocks in the macroblock foreground image; The performing region growing processing on the macroblock foreground image according to the mutual information correlation image to obtain the processed macroblock foreground image specifically includes: For each macroblock in the macroblock foreground image, when the mutual information between the macroblock and any adjacent macroblock is greater than a preset mutual information threshold, marking the adjacent macroblock as a connected domain of the macroblock; Performing mean filtering on the mutual information in each connected domain of the macroblock, and when the mean filtering value corresponding to any connected domain is greater than a preset mean threshold, performing region growing processing on the macroblock in a direction corresponding to the connected domain; The processed macroblock foreground image is obtained according to all macroblocks in the macroblock foreground image and all macroblocks processed by region growing.

2. The moving target detection method according to claim 1, characterized in that: The step of acquiring the image to be detected and detecting the motion foreground area of ​​the image to be detected by a motion detection method specifically includes: Acquire a sequence of images to be processed; wherein the sequence of images to be processed includes the image to be detected; The motion detection method is used to detect several frames of images in the image sequence to be processed to obtain the motion foreground area.

3. The moving target detection method according to claim 1, characterized in that: The method further comprises: Performing morphological processing on the motion foreground region to obtain a processed motion foreground region; Then, the macroblock division process is performed according to the image to be detected and the motion foreground area to obtain a macroblock foreground image, specifically including: A macroblock division process is performed according to the image to be detected and the processed motion foreground area to obtain a macroblock foreground image.

4. The moving target detection method according to claim 1, characterized in that: The method further comprises: Processing the mutual information correlation image by using a Bayesian filter to obtain a processed mutual information correlation image; Then, performing region growing processing on the macroblock foreground image according to the mutual information correlation image to obtain the processed macroblock foreground image specifically includes: A region growing process is performed on the macroblock foreground image according to the processed mutual information correlation image to obtain a processed macroblock foreground image.

5. The moving target detection method according to any one of claims 1 to 4, characterized in that: The acquiring of the historical timing information of each macroblock in the processed macroblock foreground image and combining with the Bayesian filter to determine whether each macroblock in the processed macroblock foreground image is a foreground specifically includes: Obtaining the historical timing information of each macroblock in the processed macroblock foreground image; wherein the historical timing information of any macroblock in the processed macroblock foreground image is expressed as [ST i (1),ST i (2),...,ST i (t-1),ST i (t)],ST i (t) represents the probability that the macroblock is determined as the foreground by the motion detection method at time t in the i-th frame image, i>0; Combined with the Bayesian filter, the probability of each macroblock in the processed macroblock foreground image being determined as the foreground is calculated; wherein the probability of any macroblock in the processed macroblock foreground image being determined as the foreground is expressed as η represents the normalization coefficient, η>0, q represents the preset time range, q>0, P(ST i (t)|fg) represents the conditional probability, and P(fg) represents the probability that the macroblock belongs to the foreground; When any macroblock in the processed macroblock foreground image is determined to be the foreground, the probability ST i When (t) is greater than a preset probability threshold, the macroblock is determined to be a foreground.

6. The moving target detection method according to claim 5, characterized in that: The method further comprises: According to the probability that each macroblock in the processed macroblock foreground image is determined to be the foreground, the probability parameter corresponding to each macroblock is updated accordingly; wherein, for any macroblock in the processed macroblock foreground image, according to ST i '(t) for the conditional probability P(ST i (t)|fg) and probability P(fg) are updated accordingly.

7. A moving target detection device, characterized in that: include: A motion foreground area acquisition module is used to acquire an image to be detected and detect the motion foreground area of ​​the image to be detected by a motion detection method; A macroblock foreground image acquisition module, used for performing macroblock division processing according to the image to be detected and the motion foreground area to obtain a macroblock foreground image; A mutual information related image acquisition module, used for calculating the mutual information between adjacent macroblocks according to the macroblock foreground image, and obtaining a mutual information related image; A macroblock foreground image processing module, used for performing region growing processing on the macroblock foreground image according to the mutual information correlation image to obtain a processed macroblock foreground image; A foreground macroblock judgment module is used to obtain the historical timing information of each macroblock in the processed macroblock foreground image, and determine whether each macroblock in the processed macroblock foreground image is a foreground by combining with a Bayesian filter; A moving target region acquisition module, used for obtaining a moving target region according to all macroblocks determined as foregrounds; The mutual information related image acquisition module specifically includes: A mutual information calculation unit, for calculating the mutual information I(a,b) between macroblock a and macroblock b according to the formula I(a,b)=H(a)+H(b)-H(a,b) for any two adjacent macroblocks a and macroblock b in the macroblock foreground image; wherein H(a) represents the entropy of macroblock a, H(b) represents the entropy of macroblock b, and H(a,b) represents the joint entropy of macroblock a and macroblock b; A mutual information related image acquisition unit, used for obtaining the mutual information related image according to the mutual information between all two adjacent macroblocks in the macroblock foreground image; The macroblock foreground image processing module specifically includes: A connected domain marking unit, for each macroblock in the macroblock foreground image, when the mutual information between the macroblock and any adjacent macroblock is greater than a preset mutual information threshold, marking the adjacent macroblock as a connected domain of the macroblock; A region growing processing unit, configured to perform mean filtering on the mutual information in each connected domain of the macroblock, and when the mean filtering value corresponding to any connected domain is greater than a preset mean threshold, perform region growing processing on the macroblock in a direction corresponding to the connected domain; The macroblock foreground image processing unit is used to obtain the processed macroblock foreground image according to all macroblocks in the macroblock foreground image and all macroblocks processed by region growth.

8. A terminal device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the moving target detection method according to any one of claims 1 to 6 when executing the computer program.

Images