Image motion detection method and device and electronic equipment

Abstract

The invention relates to an image motion detection method and device and electronic equipment, and the method comprises the steps: obtaining a long-term reference frame image and a current frame image, and enabling the long-term reference frame image to be updated through the multi-frame fusion of a historical frame image; obtaining a self-adaptive threshold value, wherein the self-adaptive threshold value is associated with the ISO of the current frame image; according to the long-term reference frame image, the current frame image, a self-adaptive threshold value and a plurality of preset window areas, obtaining the number of first motion areas; according to the first motion area number and a preset motion area number threshold value, an image motion detection result associated with the current frame image is obtained, and the image motion detection result is used for representing whether a moving shooting object appears in the video image at the moment when the current frame image is located. According to the method and the device, the detection accuracy and the wake-up rate can be ensured on the basis of low computing resources and storage resources, false alarms are reduced to the greatest extent, and the requirements of users are met in actual use.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image motion detection method, device and electronic equipment. Background Art

[0002] Currently, various video monitoring, face recognition and other technologies have been widely popularized and applied. Among these technologies, it is necessary to collect image data in real time and process and judge it to determine whether there is a motion event, so as to remind or alarm the user.

[0003] The existing means of detecting motion events mainly include image difference algorithms and background modeling algorithms. Among them, the image difference algorithm takes the absolute value by taking the difference between two frames of images, and then compares it with the pre-set threshold to judge the result. However, this algorithm has problems such as the difficulty of threshold adaptation leading to false alarms, which will greatly affect its use in actual use. The background modeling algorithm requires higher computing and storage resources, and is also unable to achieve threshold adaptation, leading to false alarms and other problems. Summary of the invention

[0004] In view of this, the present disclosure provides an image motion detection method, device, electronic device and storage medium to solve the problems of high computing and storage resources and high false alarm rate in the existing image motion detection technology.

[0005] The technical solution of the present disclosure is achieved as follows:

[0006] A method for detecting image motion, comprising:

[0007] Acquire a long-term reference frame image and a current frame image, wherein the long-term reference frame image is updated by multi-frame fusion of historical frame images, and the historical frame image and the current frame image are video frame images obtained based on original images collected in the same shooting scene;

[0008] Acquire an adaptive threshold value, where the adaptive threshold value is associated with the ISO sensitivity of the current frame image;

[0009] According to the long-term reference frame image, the current frame image, the adaptive threshold and a plurality of preset window areas, a first number of motion areas is obtained, wherein the plurality of window areas are areas for collecting content information of each local area in the video frame image, and the first number of motion areas is: the number of window areas whose content information is different when the current frame image is compared with the long-term reference frame image;

[0010] According to the first motion area number and a preset motion area number threshold, an image motion detection result associated with the current frame image is obtained, and the image motion detection result is used to characterize whether a moving object appears in the video image at the moment when the current frame image is located.

[0011] Further, obtaining the first number of motion regions according to the long-term reference frame image, the current frame image, the adaptive threshold and a plurality of preset window regions includes:

[0012] Respectively in each of the plurality of window areas, obtaining a first pixel difference absolute value according to the long-term reference frame image and the current frame image, wherein the first pixel difference absolute value is an absolute value of a pixel difference between the long-term reference frame image and the current frame image in the same window area;

[0013] A first number of motion regions is obtained according to the adaptive threshold and the first pixel difference absolute values ​​of the multiple window regions, wherein the first number of motion regions is the number of window regions satisfying a condition that the first pixel difference absolute value is greater than the adaptive threshold.

[0014] Further, obtaining the image motion detection result associated with the current frame image according to the first motion region number and a preset motion region number threshold includes:

[0015] When the first motion region number is greater than the motion region number threshold, a detection result of a moving object appearing in the current frame image is obtained.

[0016] Further, the initialized long-term reference frame image is obtained through the following process:

[0017] When acquiring the first two frames of the shooting scene, the first two frames of the image are averaged to obtain the initialized long-term reference frame image.

[0018] Further, the long-term reference frame image is updated through the following process:

[0019] When the image motion detection result is that the continuous count of the still frame image reaches a preset first trigger quantity threshold, the long-term reference frame image and the current frame image are weightedly fused using a first weight parameter group to obtain an updated long-term reference frame image; or

[0020] When the short-frame image motion detection result obtained based on the current frame image, the short-term reference frame image, the adaptive threshold, and the motion area quantity threshold is that the continuous count of the frame image stillness reaches a preset second trigger quantity threshold, the long-term reference frame image and the current frame image are weightedly fused using a second weight parameter group to obtain an updated long-term reference frame image, wherein the short-term reference frame image is the previous frame image of the current frame image, and the short-frame image motion detection result is used to characterize whether a moving subject appears in the current frame image compared with the short-term reference frame image.

[0021] Further, according to the current frame image, the short-term reference frame image, the adaptive threshold, and the motion region number threshold, a short-frame image motion detection result is obtained, including:

[0022] Respectively in each of the plurality of window areas, obtaining a second pixel difference absolute value according to the short-term reference frame image and the current frame image, wherein the second pixel difference absolute value is an absolute value of a pixel difference between the short-term reference frame image and the current frame image in the same window area;

[0023] Obtaining a second number of motion regions according to the adaptive threshold and the second pixel difference absolute values ​​of the plurality of window regions, wherein the second number of motion regions is the number of window regions satisfying a condition that the second pixel difference absolute value is greater than the adaptive threshold;

[0024] The short frame image motion detection result is obtained according to the second motion region number and the motion region number threshold.

[0025] Further, the obtaining of the adaptive threshold comprises:

[0026] According to the pre-calibrated noise model, the noise carried in the images obtained under different ISOs is obtained;

[0027] The adaptive threshold is obtained according to the noise and a preset adjustment coefficient.

[0028] Furthermore, the video frame image is a grayscale image, and the original image is a RAW format frame image;

[0029] The image motion detection method further comprises:

[0030] Acquire a RAW format frame image collected in the shooting scene;

[0031] The RAW format frame image is subjected to black level correction and gamma transformation to obtain the grayscale image.

[0032] An image motion detection device, comprising:

[0033] A frame image acquisition module is configured to acquire a long-term reference frame image and a current frame image, wherein the long-term reference frame image is updated by multi-frame fusion of historical frame images, and the historical frame image and the current frame image are video frame images obtained based on original images collected in the same shooting scene;

[0034] A threshold acquisition module is configured to execute acquisition of an adaptive threshold, where the adaptive threshold is associated with the ISO sensitivity of the current frame image;

[0035] A first motion region number obtaining module is configured to obtain a first motion region number according to the long-term reference frame image, the current frame image, the adaptive threshold and a plurality of preset window regions, wherein the plurality of window regions are regions for collecting content information of each local region in the video frame image, and the first motion region number is: the number of window regions whose content information is different when the current frame image is compared with the long-term reference frame image;

[0036] The motion detection result acquisition module is configured to execute, based on the first motion area number and a preset motion area number threshold, to obtain an image motion detection result associated with the current frame image, wherein the image motion detection result is used to characterize whether a moving object appears in the video image at the moment when the current frame image is located.

[0037] An electronic device, comprising:

[0038] processor;

[0039] a memory for storing executable instructions for the processor;

[0040] The processor is configured to execute the executable instructions to implement the image motion detection method as described in any one of the above items.

[0041] It can be seen from the above scheme that in the image motion detection method and device disclosed in the present invention, the long frame image is updated by multi-frame fusion of historical frame images, so that in the process of obtaining the first motion area number according to the long frame image, the current frame image and the adaptive threshold, the area where the shaking bushes and leaves are located will be identified as the image content presented is static, and will not be accumulated into the first motion area number. At the same time, the adaptive threshold is adjusted according to the ISO of the current frame image, thereby further reducing the interference of noise in the image, thereby reducing the false alarm rate of the image motion detection result caused by shaking objects such as shaking bushes and leaves. At the same time, in the image motion detection method and device disclosed in the present invention, the noise model of the camera itself that collects the frame image is considered, and the adaptive adjustment of the adaptive threshold according to different ISOs in the SAD process under different ISOs is completed by modeling the noise model, which effectively reduces the false detection and false alarm caused by camera noise and improves the accuracy of event detection. In addition, in the image motion detection method and device disclosed in the present invention, only the relevant calculation of SAD is used, and the fusion process of the long frame image also uses only a small amount of multiplication calculation, so compared with the prior art, the overall calculation cost and storage cost are lower. At the same time, the image motion detection method and device disclosed in the present invention only perform simple black level correction and gamma transformation preprocessing on the RAW data to obtain the required frame image in the form of a grayscale image, omitting multiple steps in the normal process of the camera ISP, and combining the low computational cost and low storage cost of the entire algorithm process to achieve low power consumption processing of image motion detection. Compared with the prior art, the technical solution disclosed in the present invention achieves low power consumption processing of image motion detection at low computational cost and low storage cost. In addition, in the image motion detection method and device of the disclosed embodiment, the SAD result of the short frame image is used to assist the update of the long frame image. The long frame image fuses the results of multiple frames to effectively reduce the noise and environmental impact carried by the single frame image. At the same time, after a long period of update, the image background information carried by the long frame image is more accurate, which can effectively improve the accuracy and wake-up rate of event detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a flow chart of an image motion detection method according to an exemplary embodiment;

[0043] Figure 2 is a flow chart showing a method of obtaining the number of first motion regions according to an exemplary embodiment;

[0044] Figure 3 is a schematic diagram showing the relationship between a long-term reference frame image, a current frame image and a window area according to an exemplary embodiment;

[0045] Figure 4is a schematic diagram of a process for obtaining a short frame motion detection result according to an exemplary embodiment;

[0046] Figure 5 is a schematic diagram of a process of obtaining an adaptive threshold according to an exemplary embodiment;

[0047] Figure 6 is a schematic diagram of an application scenario of an image motion detection method according to an exemplary embodiment;

[0048] Figure 7 is a schematic diagram of the logical structure of an image motion detection device according to an exemplary embodiment;

[0049] Figure 8 is a schematic structural diagram of another image motion detection device according to an exemplary embodiment;

[0050] Fig. 9 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0051] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below with reference to the accompanying drawings and examples.

[0052] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0053] The existing technical solutions involve the problems of complex calculations and high power consumption of the algorithms used, and the problem of high false alarm rate due to fixed threshold parameters. To address these problems, the embodiments of the present disclosure propose a solution of an image motion detection method and device as follows.

[0054] Figure 1 is a flow chart of an image motion detection method according to an exemplary embodiment. Figure 1 As shown, the image motion detection method mainly includes the following steps 101 to 104.

[0055] Step 101, obtaining a long-term reference frame image and a current frame image, wherein the long-term reference frame image is updated by multi-frame fusion of historical frame images, and the historical frame image and the current frame image are video frame images obtained based on original images collected in the same shooting scene.

[0056] In the exemplary embodiment, the long-term reference frame image refers to an image that presents long-term features in a frame image, and the long-term reference frame image is a single frame image. In the disclosed embodiment, the historical frame images are fused by multi-frame fusion, so that the long-term reference frame image can effectively reduce the noise and environmental impact carried by the single frame image. At the same time, after the long-term reference frame image has been fused and updated multiple times over a long period of time, the image background information carried in it is more accurate, which can effectively improve the accuracy and wake-up rate of event detection. For details on how to obtain and update the long-term reference frame image, please see the subsequent instructions.

[0057] In an illustrative embodiment, the current frame image refers to an image obtained by processing (such as black level, gamma transformation, etc.) the image at the current moment (such as a RAW image) in a real-time image captured by an image acquisition unit (such as a photosensitive device in a shooting device, such as an image sensor in a camera).

[0058] Step 102: Obtain an adaptive threshold value, where the adaptive threshold value is associated with the ISO (sensitivity) of the current frame image.

[0059] In the exemplary embodiment, different ISOs correspond to different adaptive thresholds. In the embodiment of the present disclosure, how to obtain the adaptive threshold is described in detail in the subsequent description.

[0060] Step 103: Obtain the first number of motion regions according to the long-term reference frame image, the current frame image, the adaptive threshold and the preset multiple window regions, wherein the multiple window regions are regions for collecting content information of each local region in the video frame image, and the first number of motion regions is: the number of window regions having different content information when the current frame image is compared with the long-term reference frame image.

[0061] Figure 2 is a flow chart showing a method of obtaining the number of first motion regions according to an exemplary embodiment. Figure 2 As shown, in the exemplary embodiment, step 103 may specifically include the following steps 201 to 202 .

[0062] Step 201, in each of the multiple window areas, obtain a first pixel difference absolute value based on the long-term reference frame image and the current frame image, wherein the first pixel difference absolute value is the pixel difference absolute value between the long-term reference frame image and the current frame image in the same window area.

[0063] In an exemplary embodiment, the window area is an area defined in the image. In an exemplary embodiment, step 201 is to calculate the first pixel difference absolute value for the long-term reference frame image and the current frame image in each window area. Figure 3 is a schematic diagram showing the relationship between a long-term reference frame image, a current frame image and a window area according to an exemplary embodiment, Figure 3 In the embodiment shown, the size of the window area is set to include 9 pixels in a 3*3 matrix, and the first pixel difference absolute value is obtained according to the sum of the absolute values ​​of the grayscale value differences of each pixel in the same window area between the long-term reference frame image and the current frame image. Figure 3In the window area, the 3*3 pixel area at the upper left corner of the long-term reference frame image and the current frame image is obtained. In the window area: the absolute value of the result of subtracting the grayscale values ​​of the pixels in the first row and the first column of the long-term reference frame image and the current frame image is obtained to obtain the absolute value of the difference in the grayscale values ​​of the pixels in the first row and the first column; the absolute value of the result of subtracting the grayscale values ​​of the pixels in the first row and the second column between the long-term reference frame image and the current frame image is obtained to obtain the absolute value of the difference in the grayscale values ​​of the pixels in the first row and the second column; and so on, the absolute value of the difference in the grayscale values ​​of all 9 pixel positions in the window area is obtained; and then, the absolute value of the difference in the grayscale values ​​of all 9 pixel positions in the window area is summed to obtain the first pixel difference absolute value. The pixel grayscale value reflects the presentation content at the pixel position. Based on this, the first pixel difference absolute value of the long-term reference frame image and the current frame image obtained in the same window area according to the embodiment of the present disclosure reflects the difference in the presentation content in the same window area between the long-term reference frame image and the current frame image. If the absolute value of the first pixel difference is 0, that is, the difference is 0, which is equivalent to that the presented content in the same window area has not changed between the long-term reference frame image and the current frame image, that is, the presented content in the same window area is static between the long-term reference frame image and the current frame image; if the absolute value of the first pixel difference is small, that is, the difference is small, which is equivalent to that the presented content in the same window area has undergone a small change between the long-term reference frame image and the current frame image, that is, the presented content in the same window area may have undergone a small displacement between the long-term reference frame image and the current frame image; if the absolute value of the first pixel difference is large, that is, the difference is large, which is equivalent to that the presented content in the same window area has undergone a large change between the long-term reference frame image and the current frame image, that is, the presented content in the same window area may have undergone a large displacement between the long-term reference frame image and the current frame image, and the presented content may even have moved to other areas of the image. Regarding judging whether the change in the presented content in the window area between the long-term reference frame image and the current frame image means that the object corresponding to the presented content has actually undergone a displacement change (such as a person walking) or is just a small shake near the same position (such as leaves swaying in the wind), it can be determined by using a corresponding threshold (such as the adaptive threshold in the embodiment of the present disclosure) and combining it with the judgment of each window area. For example, in an image, if the absolute value of the first pixel difference of more window areas has reached the adaptive threshold, it means that the displayed object in the image has undergone a position change (is moving); otherwise, it means that the displayed object in the image has not undergone a position change (is stationary).

[0064] In order to obtain the change of the display content at each detail position in the image, and thus help to accurately determine whether there is a moving object in the video image, in the exemplary embodiment, each window area is obtained by sliding a window. For example, starting from the upper left corner of the image, sliding one pixel position to the right to obtain each window area in the first row, until the upper right corner of the image obtains all the window areas of the first row; sliding one pixel position downward from the upper left corner of the image to obtain the first window area of ​​the second row, starting from the first window area of ​​the second row, sliding one pixel position to the right to obtain each window area in the second row, until the rightmost position of the second row in the frame image, to obtain all the window areas of the second row; and so on, to obtain all the window areas.

[0065] Step 202: Obtain the first motion region number according to the adaptive threshold and the absolute values ​​of the first pixel differences of the plurality of window regions, wherein the first motion region number is the number of window regions satisfying the condition that the absolute value of the first pixel difference is greater than the adaptive threshold.

[0066] The adaptive threshold is used to determine whether the display content in the window area has changed (corresponding to the possible motion of the display object in the image). In an exemplary embodiment, if the absolute value of the first pixel difference of a certain window area is greater than the adaptive threshold, it indicates that the display content of the window area has changed. Since the change of the display content means that the display object may move, the window area can be called a motion area. In step 202, the absolute value of the first pixel difference of each window area can be compared with the adaptive threshold respectively, and the number of window areas (motion areas) whose absolute value of the first pixel difference is greater than the adaptive threshold is recorded to obtain the number of first motion areas, that is, the number of first motion areas is the number of window areas that meet the condition that the absolute value of the first pixel difference is greater than the adaptive threshold.

[0067] In an exemplary embodiment, if the absolute value of the first pixel difference of a window area is less than or equal to the adaptive threshold, it indicates that the display content of the window area has not changed. Since the display content has not changed, it means that the display object may be static, so the window area can be called a static area.

[0068] Generally speaking, due to the interference of image noise and the fluctuations caused by the environmental influence of the presented content (such as swaying leaves, etc.), in the video image, even in the case of a static scene, there will be different absolute values ​​of the first pixel difference between different video frames in certain window areas. In the embodiment of the present disclosure, an adaptive threshold is used to eliminate the fluctuations between different video frames of such static scenes, and the adaptive threshold is associated with the ISO of the current frame image. Different ISOs correspond to different adaptive thresholds. Therefore, the threshold can be adaptively adjusted according to different ISOs. Compared with the prior art, the false alarm rate can be reduced, and the problem of high false alarm rate can be solved or the problem of high false alarm rate can be alleviated. In addition, in the embodiment of the present disclosure, the adaptive threshold is also obtained according to the calibrated noise model, which can further reduce the false alarm caused by fluctuations caused by environmental influences (such as swaying leaves, etc.). The specific process of obtaining the adaptive threshold according to the calibrated noise model can be seen in the subsequent description.

[0069] Step 104: obtaining an image motion detection result associated with the current frame image according to the first motion region number and a preset motion region number threshold, wherein the image motion detection result is used to indicate whether a moving object appears in the video image at the moment of the current frame image.

[0070] The first number of motion areas reflects the overall size of all motion areas in the image. If the overall size of all motion areas in the image is too small, it may represent some subtle changes in the scene presented by the image (such as leaves falling in the scene due to wind blowing) or a sudden large noise that may not be eliminated in the image. In this case, what is needed is not to generate a detection result of a moving object appearing in the current frame image, or in other words, what is needed in this case is a detection result of no moving object appearing in the current frame image; if the overall size of all motion areas in the image meets the relevant requirements, it may represent the presence of a moving object in the scene presented by the image (such as a person moving in the scene of the frame image), in which case what is needed is to generate a detection result of a moving object appearing in the current frame image.

[0071] Based on this, in an exemplary embodiment, step 104 may further include: when the number of first motion regions is greater than the motion region number threshold, obtaining a detection result of a moving object appearing in the current frame image; when the number of first motion regions is less than or equal to the motion region number threshold, obtaining an image motion detection result of a still frame image. The image motion detection result of a still frame image represents a captured object that does not appear to be moving in the video image.

[0072] Among them, the motion area number threshold represents the minimum requirement for determining whether a moving object appears in a frame image. The first motion area number being greater than the motion area number threshold indicates that the total number of motion areas has reached the requirement for considering that a moving object appears in the image, that is, the object appearing in the image has reached the requirement for determining that it is a moving object.

[0073] Since the video shooting device shoots the current scene in real time, the video is composed of a current video frame and multiple historical video frames. As time goes by, at subsequent moments, the current video frame will also become a historical video frame, and a new current video frame will be obtained by shooting at the subsequent current moment.

[0074] In an illustrative embodiment, the long-term reference frame image is updated as shooting progresses through multi-frame fusion of historical frame images. Through continuous updating of multi-frame fusion, the long-term reference frame image presents the long-term characteristics of the shot scene. This long-term characteristic strengthens the characteristics of the relatively static parts in the image shooting scene (such as static objects such as trees and houses), and weakens the characteristics of the moving parts in the image shooting scene (such as moving people, cars, objects, etc.).

[0075] Since the long-term reference frame image is updated by fusion of multiple frames of historical frame images, and there is no historical frame image at the beginning of video image acquisition, it is necessary to establish or initialize a long-term reference frame image at the beginning of video image acquisition, and update the long-term reference frame image on this basis as the video is shot. Based on this, the embodiment of the present disclosure also includes obtaining the initialized long-term reference frame image through the following process:

[0076] When acquiring the first two frames of images of the shooting scene, the first two frames of images are averaged to obtain an initialized long-term reference frame image.

[0077] In the exemplary embodiment, the first two frames of images are the first two frames of original images (such as RAW images) captured by the video capture device for capturing the scene after initialization and are obtained after relevant processing.

[0078] In an exemplary embodiment, averaging the first two frames of images may include averaging the grayscale values ​​of pixels at the same position in the first frame of image and the second frame of image. In an exemplary embodiment, the first two frames of image may be grayscale images obtained after the first two frames of original images (such as RAW images) captured by the video capture device are processed.

[0079] In the disclosed embodiment, there are two cases for updating the long-term reference frame image, one is based on the comparison between the current frame image and the long-term reference frame image, and the other is based on the short-term change of the video. In the exemplary embodiment, these two cases are judged at the same time, and the corresponding long-term reference frame image is updated when one of them is met, as shown in the following case 1 and case 2. The long-term reference frame image is updated when case 1 or case 2 is met.

[0080] Case 1: The image motion detection result is that the continuous count of the still frame image reaches a preset first trigger quantity threshold;

[0081] Case 2: The short-frame image motion detection result obtained based on the current frame image, the short-term reference frame image, the adaptive threshold, and the motion region number threshold is the case where the continuous count of the still frame image reaches the preset second trigger number threshold. Among them, the short-term reference frame image is the previous frame image of the current frame image, and the short-frame image motion detection result is used to characterize whether there is a moving object in the current frame image compared with the short-term reference frame image.

[0082] In the exemplary embodiment, when the above situation 1 or situation 2 is respectively satisfied, the relevant weight parameters used for updating the long-term reference frame image are different:

[0083] In case 1, the long-term reference frame image and the current frame image are weightedly fused using the first weight parameter group to obtain an updated long-term reference frame image; in case 2, the long-term reference frame image and the current frame image are weightedly fused using the second weight parameter group to obtain an updated long-term reference frame image.

[0084] Among them, the short frame image motion detection result reflects the difference between the current frame image and the short-term reference frame image, which is a response to short-term feature changes.

[0085] In the exemplary embodiment, the execution process of the short-frame image motion detection result obtained according to the current frame image, the short-term reference frame image, the adaptive threshold, and the motion region number threshold in case 2 is similar to the process of steps 102 to 104 in the embodiment of the present disclosure, except that the long-term reference frame image and the current frame image are processed in steps 102 to 104, while the short-term reference frame image and the current frame image are processed in case 2. Specifically, Figure 4 FIG. 1 is a flow chart showing a process of obtaining a short frame image motion detection result according to an exemplary embodiment. Figure 4 As shown, in case 2, obtaining the short frame image motion detection result according to the current frame image, the short-term reference frame image, the adaptive threshold, and the motion region number threshold may specifically include the following steps 401 to 403.

[0086] Step 401: Obtain a second pixel difference absolute value in each of the multiple window areas based on the short-term reference frame image and the current frame image, wherein the second pixel difference absolute value is the pixel difference absolute value between the short-term reference frame image and the current frame image in the same window area.

[0087] The second pixel difference absolute value is similar to the first pixel difference absolute value, and step 501 calculates the second pixel difference absolute value in each window area for the short-term reference frame image and the current frame image. Figure 3 As shown, Figure 3 The long-term reference frame image shown is changed to a short-term reference frame image.

[0088] The second pixel difference absolute value is obtained by summing the absolute values ​​of the grayscale values ​​of each pixel in the same window area between the short-term reference frame image and the current frame image. Figure 3And the long-term reference frame image is regarded as the short-term reference frame image, the window area is the 3*3 pixel area in the upper left corner of the short-term reference frame image and the current frame image, in the window area: the absolute value of the result of subtracting the grayscale values ​​between the pixels in the first row and the first column between the short-term reference frame image and the current frame image is taken to obtain the absolute value of the difference in the grayscale values ​​of the pixels in the first row and the first column; the absolute value of the result of subtracting the grayscale values ​​between the pixels in the first row and the second column between the short-term reference frame image and the current frame image is taken to obtain the absolute value of the difference in the grayscale values ​​of the pixels in the first row and the second column; and so on, the absolute value of the difference in the grayscale values ​​of all 9 pixel positions in the window area is obtained; then, the absolute value of the difference in the grayscale values ​​of all 9 pixel positions in the window area is summed to obtain the second pixel difference absolute value. The second pixel difference absolute value of the short-term reference frame image and the current frame image obtained in the same window area according to the embodiment of the present disclosure reflects the difference between the presentation content in the same window area between the short-term reference frame image and the current frame image. If the absolute value of the second pixel difference is 0, that is, the difference is 0, which is equivalent to that the presented content in the same window area has not changed between the short-term reference frame image and the current frame image, that is, the presented content in the same window area is static between the short-term reference frame image and the current frame image; if the absolute value of the second pixel difference is small, that is, the difference is small, which is equivalent to that the presented content in the same window area has undergone a small change between the short-term reference frame image and the current frame image, that is, the presented content in the same window area may have undergone a small displacement between the short-term reference frame image and the current frame image; if the absolute value of the second pixel difference is large, that is, the difference is large, which is equivalent to that the presented content in the same window area has undergone a large change between the short-term reference frame image and the current frame image, that is, the presented content in the same window area may have undergone a large displacement between the short-term reference frame image and the current frame image, and the presented content may even have moved to other areas of the image. Regarding judging whether the change in the presented content in the window area between the short-term reference frame image and the current frame image means that the object corresponding to the presented content has actually undergone a displacement change (such as a person walking) or is just a small shaking near the same position (such as leaves swaying in the wind), it can be determined through an adaptive threshold and combined with the judgment of each window area. For example, in the image, if the absolute value of the second pixel difference of more window areas has reached the adaptive threshold, it means that the displayed object in the image has undergone a position change (is moving); otherwise, it means that the displayed object in the image has not undergone a position change (is stationary).

[0089] Step 402: Obtain the number of second motion regions according to the adaptive threshold and the second pixel difference absolute values ​​of the plurality of window regions, wherein the number of second motion regions is the number of window regions satisfying the condition that the second pixel difference absolute value is greater than the adaptive threshold.

[0090] In an exemplary embodiment, if the absolute value of the second pixel difference of a certain window area is greater than the adaptive threshold, it indicates that the display content of the window area has changed. Since the change of the display content means that the displayed object may move, the window area can be called a motion area. In step 402, the absolute value of the second pixel difference of each window area can be compared with the adaptive threshold, and the number of window areas (motion areas) whose absolute value of the second pixel difference is greater than the adaptive threshold is recorded to obtain the number of second motion areas, that is, the number of second motion areas is the number of window areas that meet the condition that the absolute value of the second pixel difference is greater than the adaptive threshold.

[0091] In an exemplary embodiment, if the absolute value of the second pixel difference of a window area is less than or equal to the adaptive threshold, it indicates that the display content of the window area has not changed. Since the display content has not changed, it means that the display object may be static, so the window area can be called a static area.

[0092] Step 403: Obtain a short frame image motion detection result according to the second motion region number and the motion region number threshold.

[0093] The short-frame image motion detection result is used to indicate whether a moving object appears in the video image at the moment of the current frame image compared with the short-term reference frame image.

[0094] The second number of motion areas also reflects the overall size of all motion areas in the image. If the overall size of all motion areas in the image is too small, it may represent some subtle changes in the scene presented by the image (such as leaves blown by the wind and falling in the scene) or a large noise that may not be eliminated is suddenly generated in the image. In this case, what is needed is not to generate a detection result of a moving object in the current frame image, or in other words, what is needed in this case is a detection result of no moving object in the current frame image; if the overall size of all motion areas in the image meets the relevant requirements, it may represent the presence of a moving object in the scene presented by the image (such as a person moving in the scene of the image). In this case, what is needed is to generate a detection result of a moving object in the current frame image. Unlike the first number of motion areas, the second number of motion areas is obtained based on a short-term reference frame image.

[0095] Based on this, in an exemplary embodiment, step 403 may further include: when the number of the second motion regions is greater than the motion region number threshold, obtaining a short-frame image motion detection result in which a moving object appears in the current frame image; when the number of the second motion regions is less than or equal to the motion region number threshold, obtaining a short-frame image motion detection result in which the frame image is still. The short-frame image motion detection result in which the frame image is still indicates that the photographed object presented in the current frame image has not moved compared with the short-term reference frame image.

[0096] According to the above description, as time goes by, each current frame image corresponds to an image motion detection result and a short frame image motion detection result, based on which the number of continuous counts is equal to the number of continuous current frame images obtained as time goes by. Therefore, in the exemplary embodiment, as the image acquisition device acquires the video image of the shooting scene in real time, when the number of image motion detection results of continuous frame images that are still reaches a preset first trigger number threshold or the number of short frame image motion detection results of continuous frame images that are still reaches a preset second trigger number threshold, the long-term reference frame image is updated.

[0097] The conditions for triggering the update of the long-term reference frame image in Case 1 and Case 2 are not the same. Case 1 is based on the judgment update of the long-term reference frame image, while Case 2 is based on the judgment update of the short-term reference frame image. Therefore, the judgment criteria for the stillness of the image content are different. For this reason, the update of the long-term reference frame image is also slightly different, which is reflected in the difference between the first weight parameter group and the second weight parameter group.

[0098] In an exemplary embodiment, the first weight parameter group includes a first weight parameter w1 and a second weight parameter w2, and the second weight parameter group includes a third weight parameter w3 and a fourth weight parameter w4. The formula for obtaining the updated long-term reference frame image by weighted fusion is, for example:

[0099] A'=w1*B+w2*A

[0100] A'=w3*B+w4*A

[0101] Wherein, B represents the current frame image, A represents the long-term reference frame image, and A' represents the updated long-term reference frame image. Compared with the first weight parameter group and the second weight parameter group, in the first weight parameter group, the ratio between w2 and w1 (i.e., the value of w2 / w1) is greater than the ratio between w4 and w3 (i.e., the value of w4 / w3), that is to say: in case 1, when the image motion detection result obtained based on the long-term reference frame image is that the continuous count of the image still reaches the preset first trigger quantity threshold, when the long-term reference frame image is updated, the weight of the long-term reference frame image is greater; and in case 2, when the short-frame image motion detection result obtained based on the short-term reference frame image is that the continuous count of the image still reaches the preset second trigger quantity threshold, when the long-term reference frame image is updated, the weight of the current frame image is greater. Because, case 1 is the result obtained based on the long-term reference frame image, indicating that there is little change between the current frame image and the long-term reference frame image, and the long-term reference frame image contains the long-term static features of the image content, so these long-term static features should be retained as much as possible. Therefore, compared with case 2, the weight of the long-term reference frame image in case 1 is larger; case 2 and case 1 are to update the long-term reference frame image when any one of the conditions is met. The update of the long-term reference frame image in case 2 may be performed without satisfying the condition of case 1. In this case, there may be a situation where the image motion detection result of the moving shooting object is detected due to the appearance of a moving object in the shooting scene (for example, leaves with excessive swinging amplitude caused by instantaneous strong wind), resulting in the situation that the condition of case 1 is not met but the condition of case 2 is met. At this time, in the image, the generation of the moving object may cause the moving object to interfere with the static image content, causing the static features in the original long-term reference frame image to change significantly. Therefore, in order to be able to fuse the features that may produce larger changes (for example, leaves with larger swinging amplitude), compared with case 1, the weight of the long-term reference frame image in case 2 is smaller (corresponding to the current frame image with a larger weight).

[0102] In the exemplary embodiment, the specific values ​​of w1, w2, w3 and w4 can be set based on experience, and the first trigger quantity threshold and the second trigger quantity threshold can also be set based on experience.

[0103] Figure 5 is a schematic diagram of a process of obtaining an adaptive threshold according to an exemplary embodiment, such as Figure 5 As shown, in the exemplary embodiment, the image motion detection method further includes the following process of obtaining an adaptive threshold from step 501 to step 502.

[0104] Step 501: Obtain the noise carried in images obtained under different ISOs according to a pre-calibrated noise model.

[0105] In an exemplary embodiment, the noise model includes, for example, the definition of the following formula:

[0106] noise=(kx+b) 1 / 2

[0107] Among them, x is the pixel value, noise is the noise corresponding to the pixel value, ISO is the sensitivity of the frame image, k and b are coefficients, and k and b are the function values ​​of the cubic function of ISO, that is,

[0108] k=m1*ISO 3 +m2*ISO 2 +m3*ISO+m4

[0109] b=n1*ISO 3 +n2*ISO 2 +n3*ISO+n4

[0110] Among them, m1, m2, m3, m4, n1, n2, n3, and n4 are noise model calibration parameters. According to the above formula, the noise carried in the image under different ISO conditions can be obtained, that is, the different noises corresponding to different pixel values ​​in the image.

[0111] Step 502: Obtain an adaptive threshold according to the noise and a preset adjustment coefficient.

[0112] In an exemplary embodiment, after obtaining the noise through the noise model in step 501, the noise and the adjustment coefficient are multiplied to obtain the adaptive threshold.

[0113] It should be noted that in the above formula, noise is the function value of the independent variable x, that is, different pixel values ​​in the frame image have different noise values, and further, the size of the adaptive threshold at each ISO may have corresponding different values ​​for different pixel values ​​in the frame image. Therefore, it should be understood that in the embodiment of the present disclosure, the adaptive threshold includes the adaptive threshold for each pixel value in the image, that is, the adaptive threshold is a set of parameter values, and this set of parameter values ​​includes the adaptive threshold corresponding to each pixel value in the image.

[0114] In an exemplary embodiment, the adjustment coefficient can be set as needed, for example, the setting of the adjustment coefficient can be aimed at the highest possible accuracy of the image motion detection result and the lowest possible false alarm rate. For example, the setting of the adjustment coefficient meets the requirement of 100% detection of moving objects in the scene and the lowest possible false alarm rate under this premise. In an exemplary embodiment, different adjustment coefficients can be set for different ISO value intervals to ensure that the adaptive thresholds at different ISOs can meet the requirement of 100% detection of moving objects in the scene and the lowest possible false alarm rate under this premise.

[0115] Since the adaptive threshold is adjusted according to the change of ISO of the frame image, the embodiment of the present disclosure reduces the interference caused by noise fluctuations according to the change of ISO, thereby effectively reducing the probability of errors in the image motion detection results, and further effectively reducing the false alarm rate of alarms based on the image motion detection results.

[0116] In the exemplary embodiment, the frame images are fused and compared using grayscale values. Therefore, in the exemplary embodiment, the video frame image is a grayscale image, that is, whether it is a long-term reference frame image, a current frame image or a short-term reference frame image, they are all grayscale images. In the exemplary embodiment, the original image is a RAW format frame image. Based on this, in the exemplary embodiment, the image motion detection method of the embodiment of the present disclosure further includes: obtaining a RAW format frame image collected in the shooting scene; performing black level correction and gamma transformation on the RAW format frame image to obtain a grayscale image.

[0117] Figure 6 is a schematic diagram of an application scenario of an image motion detection method according to an exemplary embodiment. Figure 6 As shown, this application scenario mainly includes RAW2Y processing, SAD processing, judgment process, long-term reference frame image calculation process, and adaptive threshold calculation process.

[0118] The RAW2Y process is the process of converting a RAW image into a grayscale image. During the RAW2Y process:

[0119] The current RAW format frame image acquired in real time in the shooting scene is converted into the current frame image in the form of a grayscale image, which includes the black level correction and gamma conversion process.

[0120] SAD, or Sum of Absolute Differences, is the sum of absolute differences. During SAD processing:

[0121] According to the two grayscale images, the absolute value of the pixel difference between the two grayscale images in each window area is obtained; according to the absolute value of the pixel difference in each window area and the adaptive threshold, the number of motion areas is obtained, wherein the number of motion areas is the number of window areas that meet the condition that the absolute value of the pixel difference is greater than the adaptive threshold; according to the number of motion areas and the preset threshold of the number of motion areas, the image motion result is obtained.

[0122] Among them, a sliding window method is used to obtain each window area in the grayscale image.

[0123] The two grayscale images may be a long-term reference frame image and a current frame image, or the two grayscale images may be a short-term reference frame image and a current frame image, wherein the short-term reference frame image is a frame image previous to the current frame image.

[0124] The absolute value of the pixel difference is represented by cost, the adaptive threshold is represented by sad_threshold, the number of motion areas is represented by move_pixel_cnt, the threshold of the number of motion areas is represented by num_threshold, and the image motion result is represented by move_flag. Move_flag = 1 indicates that a moving object appears in the image (image motion), and move_flag = 0 indicates that no moving object appears in the image (image still). In the SAD processing process, the move_pixel_cnt of the window area that meets the condition of cost>sad_threshold is recorded, and move_flag = 1 when move_pixel_cnt>num_threshold, and move_flag = 0 when move_pixel_cnt≤num_threshold.

[0125] When the two grayscale images are a long-term reference frame image and a current frame image, during the SAD processing:

[0126] According to the long-term reference frame image and the current frame image, the first pixel difference absolute value of the long-term reference frame image and the current frame image in each window area is obtained; according to the first pixel difference absolute value of each window area and the adaptive threshold, the first motion area number is obtained; according to the first motion area number and the preset motion area number threshold, the image motion detection result is obtained, wherein the image motion detection result can be expressed as a variable long_move_flag, if the first motion area number is greater than the motion area number threshold, long_move_flag=1, if the first motion area number is less than or equal to the motion area number threshold, long_move_flag=0, wherein long_move_flag=1 indicates that a moving object appears in the current frame image, and long_move_flag=0 indicates that no moving object appears in the current frame image.

[0127] The absolute value of the first pixel difference is represented by cost1, and the number of the first motion areas is represented by move_pixel_cnt1. During the SAD processing, move_pixel_cnt1 of the window area that satisfies the condition cost1>sad_threshold is recorded, and when move_pixel_cnt1>num_threshold, long_move_flag=1, and when move_pixel_cnt1≤num_threshold, long_move_flag=0.

[0128] When the two grayscale images are a short-term reference frame image and a current frame image, during the SAD processing:

[0129] According to the short-term reference frame image and the current frame image, the second pixel difference absolute value of the short-term reference frame image and the current frame image in each window area is obtained; according to the second pixel difference absolute value of each window area and the adaptive threshold, the second motion area number is obtained; according to the second motion area number and the preset motion area number threshold, the short frame image motion detection result is obtained, wherein the short frame image motion detection result can be expressed as a variable short_move_flag, if the second motion area number is greater than the motion area number threshold, short_move_flag=1, if the second motion area number is less than or equal to the motion area number threshold, short_move_flag=0, wherein short_move_flag=1 indicates that a moving object appears in the current frame image, and short_move_flag=0 indicates that no moving object appears in the current frame image.

[0130] The absolute value of the second pixel difference is represented by cost2, and the number of the second motion areas is represented by move_pixel_cnt2. During the SAD processing, move_pixel_cnt2 of the window area that satisfies the condition cost2>sad_threshold is recorded, and when move_pixel_cnt2>num_threshold, short_move_flag=1, and when move_pixel_cnt2≤num_threshold, short_move_flag=0.

[0131] During the judgment process: when long_move_flag is 1, the image motion detection result of the moving object is output; when n consecutive long_move_flags are 0 or when m consecutive short_move_flags are 0, the long-term reference frame image calculation process is triggered to update the long-term reference frame image; wherein n and m can be set based on experience.

[0132] The long-term reference frame image calculation process mainly includes:

[0133] When acquiring the first two frames of images (grayscale images) of the shooting scene, the first two frames of images are averaged to obtain an initialized long-term reference frame image;

[0134] When n consecutive long_move_flags are 0, the long-term reference frame image is updated by the following formula:

[0135] A'=w1*B+w2*A;

[0136] Alternatively, when m consecutive short_move_flags are 0, the long-term reference frame image is updated by the following formula:

[0137] A'=w3*B+w4*A;

[0138] Among them, B represents the current frame image, A represents the long-term reference frame image, A' represents the updated long-term reference frame image, w1 is the first weight parameter, w2 is the second weight parameter, w3 is the third weight parameter, w4 is the fourth weight parameter, and w1, w2, w3 and w4 can be set according to experience.

[0139] The adaptive threshold calculation process includes:

[0140] According to the noise model of the following formula, the noise carried in the images obtained under different ISOs is obtained:

[0141] noise=(kx+b) 1 / 2

[0142] Among them, x is the pixel value, noise is the noise corresponding to the pixel value, ISO is the sensitivity of the frame image, k and b are coefficients, and k and b are the function values ​​of the cubic function of ISO, that is,

[0143] k=m1*ISO 3 +m2*ISO 2 +m3*ISO+m4

[0144] b=n1*ISO 3 +n2*ISO 2 +n3*ISO+n4

[0145] Among them, m1, m2, m3, m4, n1, n2, n3, and n4 are noise model calibration parameters. According to the above formula, the noise carried in the image under different ISOs can be obtained, that is, the different noises corresponding to different pixel values ​​in the image. Among them, each calibration parameter can be set according to experience;

[0146] An adaptive threshold is obtained according to the noise and a preset adjustment coefficient.

[0147] The noise intensity is different under different ISO. Generally speaking, the larger the ISO, the greater the noise. If only the noise is used as the adaptive threshold, the error rate of motion judgment may be too high, resulting in a high probability of wrong detection results. The adaptive threshold obtained by correcting the noise with an appropriate adjustment coefficient can reduce the error rate of judgment and improve the accuracy of the image motion detection result. In an illustrative embodiment, the product of the noise and the adjustment coefficient can be determined as the adaptive threshold. Among them, the adjustment coefficient can be set according to experience or needs.

[0148] Figure 7 is a schematic diagram of the logical structure of an image motion detection device according to an exemplary embodiment. Figure 7 As shown, the image motion detection method and device includes a frame image acquisition module 701, a threshold acquisition module 702, a first motion region quantity acquisition module 703 and a motion detection result acquisition module 704.

[0149] The frame image acquisition module 701 is configured to execute the acquisition of a long-term reference frame image and a current frame image, wherein the long-term reference frame image is updated by multi-frame fusion of historical frame images, and the historical frame image and the current frame image are video frame images obtained based on the original images collected in the same shooting scene.

[0150] The threshold acquisition module 702 is configured to acquire an adaptive threshold, where the adaptive threshold is associated with the ISO of the current frame image.

[0151] The first motion region number acquisition module 703 is configured to execute, based on the long-term reference frame image, the current frame image, an adaptive threshold and a plurality of preset window areas, to obtain the first motion region number, wherein the plurality of window areas are areas for collecting content information of each local area in the video frame image, and the first motion region number is: the number of window areas having different content information when the current frame image is compared with the long-term reference frame image.

[0152] The motion detection result acquisition module 704 is configured to execute, based on the first motion area number and a preset motion area number threshold, to obtain an image motion detection result associated with the current frame image, and the image motion detection result is used to characterize whether a moving object appears in the video image at the moment when the current frame image is located.

[0153] In an exemplary embodiment, the first motion area number obtaining module 703 is further configured to perform: obtaining a first pixel difference absolute value in each of the multiple window areas according to the long-term reference frame image and the current frame image, wherein the first pixel difference absolute value is the pixel difference absolute value between the long-term reference frame image and the current frame image in the same window area; obtaining the first motion area number according to an adaptive threshold and the first pixel difference absolute values ​​of the multiple window areas, wherein the first motion area number is the number of window areas that satisfy the condition that the first pixel difference absolute value is greater than the adaptive threshold.

[0154] In the exemplary embodiment, the motion detection result obtaining module 704 is further configured to execute: when the number of the first motion regions is greater than the motion region number threshold, obtaining a detection result of a motion object appearing in the current frame image.

[0155] In an exemplary embodiment, the image motion detection device further includes:

[0156] The long-term reference frame image initialization module is configured to execute, when acquiring the first two frames of images of the shooting scene, averaging the first two frames of images to obtain an initialized long-term reference frame image.

[0157] In an exemplary embodiment, the image motion detection device further includes:

[0158] The long-term reference frame image update module is configured to perform:

[0159] When the image motion detection result is that the continuous count of still frame images reaches a preset first trigger quantity threshold, the long-term reference frame image and the current frame image are weightedly fused using the first weight parameter group to obtain an updated long-term reference frame image; or, when the short-frame image motion detection result obtained based on the current frame image, the short-term reference frame image, the adaptive threshold, and the motion area quantity threshold is that the continuous count of still frame images reaches a preset second trigger quantity threshold, the long-term reference frame image and the current frame image are weightedly fused using the second weight parameter group to obtain an updated long-term reference frame image, wherein the short-term reference frame image is the previous frame image of the current frame image, and the short-frame image motion detection result is used to characterize whether a moving subject appears in the current frame image compared with the short-term reference frame image.

[0160] In an exemplary embodiment, the long-term reference frame image update module is further configured to perform:

[0161] In each of the plurality of window areas, respectively, a second pixel difference absolute value is obtained according to the short-term reference frame image and the current frame image, wherein the second pixel difference absolute value is an absolute value of a pixel difference between the short-term reference frame image and the current frame image in the same window area;

[0162] Obtaining the number of second motion regions according to the adaptive threshold and the absolute values ​​of the second pixel differences of the plurality of window regions, wherein the number of the second motion regions is the number of window regions satisfying a condition that the absolute value of the second pixel difference is greater than the adaptive threshold;

[0163] According to the second motion region number and the motion region number threshold, a short frame image motion detection result is obtained.

[0164] In an exemplary embodiment, the image motion detection device further includes:

[0165] The adaptive threshold calculation module is configured to execute: obtaining the noise carried in the images obtained under different ISOs according to a pre-calibrated noise model; and obtaining the adaptive threshold according to the noise and a preset adjustment coefficient.

[0166] In an illustrative embodiment, the video frame image is a grayscale image, and the original image is a RAW format frame image; the image motion detection device also includes: an image processing module, which is configured to execute: obtaining a RAW format frame image collected in a shooting scene; performing black level correction and gamma transformation on the RAW format frame image to obtain a grayscale image.

[0167] Figure 8 is a schematic structural diagram of another image motion detection device according to an exemplary embodiment. Figure 8 As shown, the image motion detection device includes an image acquisition unit 801 and a motion detection unit 802 .

[0168] The image acquisition unit 801 is used to acquire the original image in the shooting scene.

[0169] The motion detection unit 802 is electrically connected to the image acquisition unit 801 and is used to perform:

[0170] A long-term reference frame image and a current frame image are obtained, wherein the long-term reference frame image is updated by multi-frame fusion of historical frame images, and the historical frame image and the current frame image are video frame images obtained based on original images collected in the same shooting scene; an adaptive threshold is obtained, and the adaptive threshold is associated with the ISO of the current frame image; a first motion region number is obtained based on the long-term reference frame image, the current frame image, the adaptive threshold and a plurality of preset window regions, wherein the plurality of window regions are regions for collecting content information of each local region in the video frame image, and the first motion region number is: the number of window regions whose content information is different when the current frame image is compared with the long-term reference frame image; an image motion detection result associated with the current frame image is obtained based on the first motion region number and a preset motion region number threshold, and the image motion detection result is used to characterize whether a moving shooting object appears in the video image at the moment when the current frame image is located.

[0171] In the fields of monitoring and image motion detection, the purpose of monitoring an area is to care about the movement of people or objects related to people (such as pets, cars, etc.) in the area, rather than the shaking of leaves and flowers. If the alarm is frequently triggered by the shaking of leaves and flowers during the motion detection process, its use will be greatly affected. In the image motion detection method and device of the disclosed embodiment, the long-term reference frame image is updated by multi-frame fusion of historical frame images, so that areas such as bushes and leaves are constantly fused, so that the features of the areas where the shaking bushes and leaves are located in the long-term reference frame image are constantly homogenized, so that in the process of obtaining the first number of motion areas according to the long-term reference frame image, the current frame image and the adaptive threshold, the area where the shaking bushes and leaves are located will be identified as the image content presented is static, and will not be accumulated into the first number of motion areas. At the same time, the adaptive threshold is adjusted according to the ISO of the current frame image, thereby further reducing the interference of noise in the image, thereby reducing the false alarm rate of the image motion detection result caused by shaking objects such as shaking bushes and leaves.

[0172] In the image motion detection method and device of the disclosed embodiments, the noise model of the camera itself that captures the frame image is taken into consideration. By modeling the noise model, the adaptive threshold in the SAD process under different ISOs is adaptively adjusted according to different ISOs, thereby effectively reducing false detections and false alarms caused by camera noise and improving the accuracy of event detection.

[0173] In the disclosed embodiment, only the SAD calculation between the long-term reference frame image and the current frame image and the SAD calculation between the short-term reference frame image and the current frame image are used, and the fusion process also uses only a small amount of multiplication calculations. Therefore, compared with the prior art, the overall computational cost and storage cost are lower. At the same time, the image motion detection method and device of the disclosed embodiment only performs simple black level correction and gamma transformation preprocessing on the RAW data to obtain the required frame image in the form of a grayscale image, omitting multiple steps in the normal process of the camera ISP (Image Signal Process), combined with the low computational cost and low storage cost of the entire algorithm process, to achieve low power consumption processing of image motion detection.

[0174] Compared with the embodiments of the present disclosure, the existing technologies have the problems of complex calculations and the inability to adapt the thresholds. For example, in an existing solution, large errors are likely to occur at the corner points when the image moves violently, and it is easy to fail in a high ISO and high noise environment, and the calculation is complex and consumes a lot of power. Moreover, its threshold is a pre-set fixed threshold that cannot adapt to complex situations in different environments and at different ISOs. Alternatively, some existing solutions use long-term video sequences as input, which uses too many computing resources and storage resources, is not conducive to the algorithm implantation of low-power small devices, and does not achieve threshold adaptation and cannot cope with complex situations in different ISOs and environments. It can be seen that compared with these existing technologies, the technical solution of the present disclosure achieves low-power processing of image motion detection at low computing cost and low storage cost.

[0175] In addition, in the image motion detection method and device of the disclosed embodiment, the SAD result of the short-term reference frame image is used to assist the update of the long-term reference frame image. The long-term reference frame image combines the results of multiple frames to effectively reduce the noise and environmental impact carried by the single frame image. At the same time, after a long period of update, the image background information carried by the long-term reference frame image is more accurate, which can effectively improve the accuracy and wake-up rate of event detection.

[0176] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present disclosure, and will not be described in detail here.

[0177] Regarding the image motion detection device in the above embodiment, the specific manner in which each unit performs operations has been described in detail in the embodiment of the image motion detection method, and will not be elaborated here.

[0178] It should be noted that: the above embodiment only illustrates the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0179] Fig. 9It is a structural diagram of an electronic device provided by an embodiment of the present disclosure. In some embodiments, the electronic device is a server. The electronic device 900 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 901 and one or more memories 902, wherein at least one program code is stored in the memory 902, and the at least one program code is loaded and executed by the processor 901 to implement the image motion detection method provided by the above-mentioned various embodiments. Of course, the electronic device 900 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The electronic device 900 may also include other components for implementing device functions, which will not be repeated here.

[0180] In an exemplary embodiment, a computer-readable storage medium including at least one instruction is also provided, such as a memory including at least one instruction. The at least one instruction can be executed by a processor in a computer device to complete the image motion detection method in the above embodiment.

[0181] Optionally, the above-mentioned computer-readable storage medium may be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium may include ROM (Read-Only Memory), RAM (Random-Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic tape, floppy disk and optical data storage device, etc.

[0182] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. A method for detecting image motion, comprising: Acquire a long-term reference frame image and a current frame image, wherein the long-term reference frame image is updated by multi-frame fusion of historical frame images, and the historical frame image and the current frame image are video frame images obtained based on original images collected in the same shooting scene; Acquire an adaptive threshold value, where the adaptive threshold value is associated with the ISO sensitivity of the current frame image; According to the long-term reference frame image, the current frame image, the adaptive threshold and a plurality of preset window areas, a first number of motion areas is obtained, wherein the plurality of window areas are areas for collecting content information of each local area in the video frame image, and the first number of motion areas is: the number of window areas whose content information is different when the current frame image is compared with the long-term reference frame image; According to the first motion area number and a preset motion area number threshold, an image motion detection result associated with the current frame image is obtained, and the image motion detection result is used to characterize whether a moving object appears in the video image at the moment when the current frame image is located.

2. The image motion detection method according to claim 1, characterized in that: The obtaining of the first motion region quantity according to the long-term reference frame image, the current frame image, the adaptive threshold and a plurality of preset window regions comprises: Respectively in each of the plurality of window areas, obtaining a first pixel difference absolute value according to the long-term reference frame image and the current frame image, wherein the first pixel difference absolute value is an absolute value of a pixel difference between the long-term reference frame image and the current frame image in the same window area; A first number of motion regions is obtained according to the adaptive threshold and the first pixel difference absolute values ​​of the multiple window regions, wherein the first number of motion regions is the number of window regions satisfying a condition that the first pixel difference absolute value is greater than the adaptive threshold.

3. The image motion detection method according to claim 1, characterized in that: The obtaining, according to the first motion region number and a preset motion region number threshold, an image motion detection result associated with the current frame image includes: When the first motion region number is greater than the motion region number threshold, a detection result of a moving object appearing in the current frame image is obtained.

4. The image motion detection method according to claim 1, characterized in that: The initialized long-term reference frame image is obtained by the following process: When acquiring the first two frames of the image of the shooting scene, the first two frames of the image are averaged to obtain the initialized long-term reference frame image.

5. The image motion detection method according to claim 1, characterized in that: The long-term reference frame image is updated by the following process: When the image motion detection result is that the continuous count of the still frame image reaches a preset first trigger quantity threshold, the long-term reference frame image and the current frame image are weightedly fused using a first weight parameter group to obtain an updated long-term reference frame image; or When the short-frame image motion detection result obtained based on the current frame image, the short-term reference frame image, the adaptive threshold, and the motion area quantity threshold is that the continuous count of the frame image stillness reaches a preset second trigger quantity threshold, the long-term reference frame image and the current frame image are weightedly fused using a second weight parameter group to obtain an updated long-term reference frame image, wherein the short-term reference frame image is the previous frame image of the current frame image, and the short-frame image motion detection result is used to characterize whether a moving subject appears in the current frame image compared with the short-term reference frame image.

6. The image motion detection method according to claim 5, characterized in that: Obtaining a short frame image motion detection result according to the current frame image, the short-term reference frame image, the adaptive threshold, and the motion region quantity threshold, including: Respectively in each of the plurality of window areas, obtaining a second pixel difference absolute value according to the short-term reference frame image and the current frame image, wherein the second pixel difference absolute value is an absolute value of a pixel difference between the short-term reference frame image and the current frame image in the same window area; Obtaining a second number of motion regions according to the adaptive threshold and the second pixel difference absolute values ​​of the plurality of window regions, wherein the second number of motion regions is the number of window regions satisfying a condition that the second pixel difference absolute value is greater than the adaptive threshold; The short frame image motion detection result is obtained according to the second motion region number and the motion region number threshold.

7. The image motion detection method according to claim 1, characterized in that: The obtaining of the adaptive threshold comprises: According to the pre-calibrated noise model, the noise carried in the images obtained under different ISOs is obtained; The adaptive threshold is obtained according to the noise and a preset adjustment coefficient.

8. The image motion detection method according to claim 1, characterized in that: The video frame image is a grayscale image, and the original image is a RAW format frame image; The image motion detection method further comprises: Acquire a RAW format frame image collected in the shooting scene; The RAW format frame image is subjected to black level correction and gamma transformation to obtain the grayscale image.

9. An image motion detection device, characterized in that: include: A frame image acquisition module is configured to acquire a long-term reference frame image and a current frame image, wherein the long-term reference frame image is updated by multi-frame fusion of historical frame images, and the historical frame image and the current frame image are video frame images obtained based on original images collected in the same shooting scene; A threshold acquisition module is configured to acquire an adaptive threshold, where the adaptive threshold is associated with the ISO sensitivity of the current frame image; A first motion region number obtaining module is configured to obtain a first motion region number according to the long-term reference frame image, the current frame image, the adaptive threshold and a plurality of preset window regions, wherein the plurality of window regions are regions for collecting content information of each local region in the video frame image, and the first motion region number is: the number of window regions whose content information is different when the current frame image is compared with the long-term reference frame image; The motion detection result acquisition module is configured to execute, based on the first motion area number and a preset motion area number threshold, to obtain an image motion detection result associated with the current frame image, wherein the image motion detection result is used to characterize whether a moving object appears in the video image at the moment when the current frame image is located.

10. An electronic device, characterized in that: include: processor; a memory for storing executable instructions for the processor; The processor is configured to execute the executable instructions to implement the image motion detection method according to any one of claims 1 to 8.

Images