A motion detection method for determining motion within a scene and a system for determining motion within a scene

Abstract

The present disclosure relates to a motion detection method (100) for determining motion within a scene comprising one image defined by a plurality of pixels organized in a matrix comprising a number of rows (r1,r2,…,rn), and a number of columns (c1,c2,…,cm); wherein the method (100) comprises the following steps; capturing (102) the image by obtaining pixel- value from each pixel of the image in a pre-determined sequential order such that each pixel- value is obtained at a different period of time; analyzing (104) each pixel-value by comparing said pixel-value with a pixel-value of at least one neighboring pixel, and determining (106), based on the analysis, if an object within said scene is moving. The disclosure further relates to a system (300) for determining motion within a scene.

Description

[0001] A motion detection method for determining motion within a scene and a system for determining motion within a scene

[0002] Technical field

[0003] The present disclosure relates to a motion detection method for determining motion within a scene and a system for determining motion within a scene. More specifically, the disclosure relates to a motion detection method for determining motion within a scene and a system for determining motion within a scene as defined in the introductory parts of the independent claims.

[0004] Background art

[0005] Conventional methods and systems for motion detection, e.g., for determining motion of an object within a scene of interest, use at least two images. Said images are compared to each other in order to determine if the object within said image is in motion or not.

[0006] A problem with the solutions of the prior art is that for a platform that moves fast at low height there might be no or too small overlap between the images, i.e., a motion is hard to detect, or even not detected at all. Further, fast moving airborne objects may also be difficult to detect using said images.

[0007] There is thus a need for a method and a system for determining motion when the system and an object have a high velocity in relation to each other.

[0008] Summary

[0009] It is an objective of the present disclosure to mitigate, alleviate or eliminate one or more of the above-identified deficiencies and disadvantages in the prior art and solve at least the above mentioned problem.

[0010] The present invention relates to a method of detecting relative motion, i.e., motion between a moving platform and the ground, between a stationary sensor and a moving object, or between two moving platforms.

[0011] According to a first aspect there is provided a motion detection method for determining a motion of an object within one image comprising a plurality of pixels organized in a matrix comprising a number of rows, and a number of columns; wherein the method comprises the following steps; capturing the image by obtaining a pixel-value from each pixel of the image in a pre-determined sequential order such that each pixel-value is obtained at a different time than pixel-values of neighboring pixels; analyzing each pixel-value by comparing the pixelvalue with a pixel-value of at least one neighboring pixel, and determining, based on the analysis, if the object within the image is moving.

[0012] In other words, the method uses only one single image covering a scene in order to detect a motion of an object within said scene. Using only one image to determine motion utilizes the fact that the image (also commonly referred to as frame) is captured by rapidly scanning across the scene, which results in a plurality of pixels each captured in slightly different times (similar to a rolling shutter) which differs from taking a snapshot of the entire scene at one single instant in time (known as global shutter). The method consequently utilizes the fact that it takes time to capture the image, i.e., the method takes advantages of the changes that take place within the scene during the time it takes to capture the image.

[0013] The step of analyzing each pixel-value considers analyzing the differences between pixel-value of a pixel and pixel-values of the neighboring pixels.

[0014] In an example embodiment, the obtained pixel-value of a pixel is a set of pixel-values, i.e., a pixel may comprise a plurality of pixel-values. Example of pixel-values may be, but not limited to, color intensity, light, and / or distance.

[0015] The number of neighboring pixels used in the step of analyzing can be adjusted and altered in different cases. A neighboring pixel is to be interpreted as a pixel adjacent with respect to space and / or time. The term adjacent with respect to time may be referred to as obtained nearest before and / or nearest after in time with regard to the obtaining of the pixel value of a regarded pixel, according to the sequence of obtaining of pixel values.

[0016] In an example, for a matrix comprising rows and columns, wherein each pixel has the shape of a square, a pixel can be determined to have up to four (4) neighboring pixels, up- down-, right- , and left. In another example, a matrix comprising rows and columns, wherein each pixel has the shape of a square, a pixel can be determined to have up to eight (8) neighboring pixels, up- , down-, right- and left, as well as additional four neighboring pixels in diagonal direction, i.e., up left, up right, down left and down right.

[0017] In another example, for a matrix comprising rows and columns, wherein each pixel has the shape of a hexagonal, said pixel can be determined to have up to six (6) neighboring pixels, up right, up left, right, left, down right, and down left.

[0018] As stated above, the number of neighboring pixels used in the step of analyzing can be adjusted and altered in different scenarios, i.e., a neighboring pixel can therefore be interpreted as a pixel adjacent with respect to space and / or time relevant to the specific scenario. The term adjacent with respect to time may be interpreted as the pixel located nearest before and / or nearest after in time in relation to the analyzed pixel.

[0019] The method has the advantage of having a fast response time and low computational complexity. The low computational complexity contributes to the low latency of the response. A low computational complexity also means that it is feasible to use the method in platforms where heat produced as a side effect of computation would be a problem.

[0020] It is possible to start the processing as soon as a local neighbourhood in the image has become available, even before a full frame has been exposed, i.e., before a full image has been captured. Thereby a fast response can be achieved.

[0021] According to some embodiments, the step of obtaining a pixel-value from each pixel comprises obtaining pixel-value from each pixel of a first row starting at the pixel located at the first column and moving in a first direction towards the end column of the first row; and obtaining pixel-value from each pixel of a second row starting at the pixel located at the end column and moving in a second direction towards the first column of the second row.

[0022] In other words, the pre-determined sequential order is a pattern, which captures pixel-values in an alternating row-wise manner. For example, in an image in which the pixels are organized in a matrix comprising rows and columns, the method capturing pixel-values from each pixel in the first row, starting with the pixel in the first column, and followed by the pixel in the second column, and then the pixel in the third column, and so on until the last pixel on the first row located in the last column has been captured. The scanning direction is then changed to a second direction opposite to the first direction and pixel-values are captured from each pixel in the second row, starting with the pixel in the last column, and followed by the pixel in the penultimate last column, and so on until the first pixel on the second row located in the first column has been captured. In this example, the pre-determined row-wise alternating pattern may be described as a snake-like pattern or meander pattern.

[0023] The advantage of capturing pixel-values from pixels closely located within the matrix is that it improves the analyzing process with respect to at least response time and / or accuracy. For example, the fast response is a result of the analysing process being able to start as soon as neighbouring pixel-values of the image have become available, i.e., even before the full image has been captured.

[0024] According to to some embodiments, a neighboring pixel is a pixel whose value has been obtained nearest before in time and / or nearest after in time in relation to the currently obtained pixel.

[0025] According to some embodiments, the analyzing is performed utilizing fast Fourier transform, FFT.

[0026] According to a second aspect there is provided a system for determining motion within a scene comprising one image defined by a plurality of pixels organized in a matrix comprising a pre-defined number of rows, and a pre-determined number of columns, wherein the system comprises; a camera system configured to capture the image; an analyzing device configured to analyze positions in the image by; a determining device configured to determine if there is relative motion between the object corresponding to the position in the image and the imaging device.

[0027] Effects and features of the second aspect are to a large extent analogous to those described above in connection with the first aspect. Embodiments mentioned in relation to the first aspect are largely compatible with the second aspect.

[0028] The present disclosure will become apparent from the detailed description given below. The detailed description and specific examples disclose preferred embodiments of the disclosure by way of illustration only. Those skilled in the art understand from guidance in the detailed description that changes and modifications may be made within the scope of the disclosure. Hence, it is to be understood that the herein disclosed disclosure is not limited to the particular component parts of the device described or steps of the methods described since such device and method may vary. It is also to be understood that the terminology used herein is for purpose of describing particular embodiments only, and is not intended to be limiting. It should be noted that, as used in the specification and the appended claims, the articles "a", "an", "the", and "said" are intended to mean that there are one or more of the elements unless the context explicitly dictates otherwise. Thus, for example, reference to "a unit" or "the unit" may include several devices, and the like. Furthermore, the words "comprising", "including", "containing" and similar wordings does not exclude other elements or steps.

[0029] Brief descriptions of the drawings

[0030] The above objects, as well as additional objects, features and advantages of the present disclosure, will be more fully appreciated by reference to the following illustrative and nonlimiting detailed description of example embodiments of the present disclosure, when taken in conjunction with the accompanying drawings.

[0031] Figure 1 shows a flow chart of the method according to an embodiment of the present disclosure.

[0032] Figure 2a-2d shows illustrative views of the method according to an embodiment of the present disclosure.

[0033] Figure 3a depicts a block diagram depicting embodiments of a system according to an embodiment of the present disclosure.

[0034] Figure 3b depicts schematically a system according to an embodiment of the present disclosure.

[0035] Figure 4a-4c depict graphical illustrations according to an embodiment of the present disclosure.

[0036] Figure 5a-5c depict graphical illustrations according to an embodiment of the present disclosure. Detailed description

[0037] The present disclosure will now be described with reference to the accompanying drawings, in which preferred example embodiments of the disclosure are shown. The disclosure may, however, be embodied in other forms and should not be construed as limited to the herein disclosed embodiments. The disclosed embodiments are provided to fully convey the scope of the disclosure to the skilled person.

[0038] Figure 1 shows a flow chart of an example of a motion detection method 100.

[0039] The motion detection method 100 is suitable for determining motion of an object within one image. As can be seen in Fig. 2a-2c, the image comprises a plurality of pixels organized in a matrix which comprises a number of rows ri,r2,...,rn, and a number of columns ci,C2,...,cm. The method 100 comprises the following steps.

[0040] In a first step, the image is captured 101 by obtaining 102 at least one pixel-value from each pixel of the image. The image capturing process is performed in a pre-determined sequential order such that each pixel-value is obtained at a different time than pixel-values of neighboring pixels, i.e., neighboring pixel-values are not captured at the exact same times. Examples of the pre-determined sequential order can be seen in Fig. 2a-2c.

[0041] Thereafter, each pixel-value is analyzed 104 by comparing the pixel-value with a pixel-value of at least one neighboring pixel.

[0042] In an example, the step of analyzing 104 comprises performing a one-dimensional phasecorrelation between parts of two consecutive pixel rows. The parts are obtained by windowing of the pixel values on the rows around the desired column index. Followed by comparing, from the phase correlation, the position of where the maximum correlation is to the positions obtained similarly for adjacent rows. When there is relative motion in the scene, there will be a systematic difference in the position of max correlation on even rows when doing a correlation with its subsequent row, and the position of max correlation position on odd rows when doing a correlation with its subsequent row. However, an exception to this exists for the middle column, where the times will be equally long before returning to the same column, thereby making the analysis impractical in this area. However, for spatial positions other than the center column, it will be possible to obtain the angular velocity of the relative motion using the differences of the values of the max correlation position when the correlation is performed starting with either an even or an odd row, and using the distance from center column as part of the input. The comparison is performed by comparing for instance color intensity, light, and / or distance.

[0043] In images there is typically large similarities between the pixels on a row and its neighboring rows, e.g., the previous rows and the next rows. In order to further disclose an advantage of the invention, let us consider a case, depicted in Fig. 2d, which shows an image 200 comprising a vertical structure 220. As a starting point, assume that the rows in the image 200 are captured row-wise, e.g., whole rows of pixels in the image 200 are exposed simultaneously, starting at the top row ri of the image 200 and progressing down towards the last row rn. If there is no relative motion between the image 200 and the camera system configured to capture the image 200, and assuming that all geometrical distortion in the camera has been eliminated, and also assuming that the noise is zero, all the rows will be identical. However, if there is a constant motion between the imaging device and the scene (this example is not shown), there will be a shift in image content from one row to the next row, and the shift is equally large for all rows, only depending on the relative speed of the imaging device and the scene, and impossible to distinguish from the image of a slanted structure captured without any relative motion. So, if instead the scanning pattern is that of Fig. 2a, the row content shift will depend not only on the relative motion, but also on whether the line is even or odd, and which columns the pixels belong to. Since row evenness and horizontal position (i.e., column) are known, the image can be analyzed locally to detect and measure the relative motion. The analysis may take into account that the image signal, such as depicted in Fig. 4a, may become more or less stretched spatially on even and odd rows.

[0044] The method further comprises a step of determining 106, based on the analysis, if an object within the image is moving or not.

[0045] In an example illustrated in Fig.2a, the step of obtaining pixel-values from each pixel comprises 102a the step of obtaining pixel-values from each pixel pi,i- pi,mof a first row ri .The pixelvalue obtaining process starts at the pixel located on the first column ci of the first row ri and moves on in a first direction to an end column cmof the first row ri. In other words, the obtaining process steps through the row pixel by pixel and obtains pixel-values from each pixel at a slightly different time, i.e., the pixel-value of neighboring pixels are obtained at different times. The pixel-value obtaining process proceeds by obtaining 102b pixel-value from each pixel pz.m - P2 of the second row r2 now starting at an end column cmand moving on, in a second direction opposite to the first direction, to the first column ci of the second row r2. The first row riand the second row r2 are adjacent to each other.

[0046] In the example embodiment illustrated in Fig.2a, pixel Pl is considered to have one neighboring pixel P2. Pixel P4 is considered to have two neighboring pixels P3 and P5. In another example, pixel P4 may be considered to have three neighboring pixels P3, P5 and P6 that are located close in time and space. The terms "close in time" and "neighboring in time" may be referred to as obtained nearest before and nearest after in time with regard to the obtaining of the pixel value of a regarded pixel, according to the sequence of obtaining of pixel values.

[0047] In another example embodiment illustrated in Fig.2b, the pre-determined pattern is changed in comparison to the example shown in Fig. 2a. The scanning pattern is now column-wise instead of row-wise. The step of obtaining pixel-values from each pixel comprises 102a the step of obtaining pixel-values from each pixel pi,i- pn,i of a first column ci starting at a first row ri and moving on, in a first direction, to an end row rnof the first column ci. Followed by obtaining 102b pixel-value from each pixel pn,2 - pi,2 of a second column C2 starting at an end row rnand moving on, in a second direction opposite to the first direction, to the first row ri of the second column C2. The first column ci and the second column C2 are adjacent to each other.

[0048] Figure 2c shows an embodiment in which the matrix comprising rows and columns, in which the pixels have the shape of hexagon. The step of obtaining pixel-values from each pixel follow the same principle as described in relation to Fig.2a-2b, i.e., the capturing process may be performed row-wise as well as column-wise.

[0049] In an example, the step of analyzing 104 is performed utilizing Fourier transform, e.g., Fast Fourier Transform, FFT.

[0050] Figure 3a shows an illustrative block diagram of the system according to an embodiment of the present disclosure. The second aspect of this disclosure shows a system 300 for determining motion within one image. The system 300 comprises a plurality of pixels organized in a matrix comprising a number of rows ri,r2,...,rn, and a number of columns ci,C2,...,cm. The system 300 comprises a camera system 302 configured to capture the image; an analyzing device 304 configured to analyze positions in the image; a determining device 306 configured to determine if an object 310 (illustrated in Fig. 3b) within the scene is moving and a control unit 308 configured to execute the method according to any of the embodiments of the first aspect.

[0051] Figure 3b shows an illustrative view of a system 300 according to an embodiment of the present disclosure. The figure aims to illustrate that the system 300 may be applied to a vehicle 312, e.g. an airplane.

[0052] Fig. 4a-4c show the image signal on one row of the image, a windowing function, and the result of applying the windowing function to the signal respectively. Horizontal scanning of the image is assumed, as shown in Fig. 2a. The indexes on the horizontal axes are pixel indexes from 1 to m, pixel 1 is the leftmost pixel and the pixel with the highest index is the rightmost pixel on the row. In this particular example, m equals 308.

[0053] Fig. 4a depicts schematically how pixel values, such as intensity or distance, can vary on an image row. The pixels from an imaging device are typically obtained as discrete values, e.g., integers in the digital range 0-255, or 0-65535. In this particular example 0-255. The pixel values are obtained by imaging a physical reality which is typically at least piece-wise continuous, giving the signal an at least piece-wise spatial continuity over the image row. It is advantageous that the pixel values lie in the range strictly between the minimum value and the maximum value of the digital range, so that saturation does not occur. If the physical signal would have an amplitude causing the digital signal to be at minimum or maximum value for more than one consecutive pixel, the analysis could become unreliable. In Fig. 4a, the signal lies between the integer values of 0 and 255 for all pixels on the row.

[0054] Fig. 4b depicts a windowing function that can be used to extract a part of an image row by pixel-wise multiplication of values. The purpose of windowing is to make the analysis local over the image row, thus enabling different motions to be detected on different parts of the image row by applying windows centred around different positions on the row. Characteristic of a windowing function is that its value is 1 for the spatial position being analysed, and 0 for positions far from this position. Windowing functions are also typically symmetric around the position being analysed.

[0055] Fig. 4c shows the result of applying the windowing function to an image row by pixel-wise multiplication. The value for each pixel position in Fig. 4c is obtained by multiplying the signal value for each pixel on the row (as depicted in Fig. 4a) with the value for the same pixel of the windowing function (as depicted in Fig. 4b).

[0056] Fig. 5a shows the image signal from two consecutive lines. The solid line shows the same row as the one shown in Fig. 4a, and the dotted line shows the image signal from the next line. The two lines are very similar and overlap almost entirely. In this example, there is no relative motion between the camera and the scene.

[0057] Fig. 5b shows the windowed version of the two image row signals from Fig. 5a. The window function used in this example is the one shown in Fig. 4b.

[0058] Fig. 5c shows the result of phase correlation between the two windowed image row signals. The horizontal axis shows the relative displacement between the two image row signals. Since there is no relative motion between the scene and the camera in this example the correlation peak value appears for the value 0 displacement. How a phase correlation is performed, including how to enhance the resolution to sub-pixel levels, is described in the signal processing literature.

[0059] The person skilled in the art realizes that the present disclosure is not limited to the preferred embodiments described above. The person skilled in the art further realizes that modifications and variations are possible within the scope of the appended claims. For example, horizontally and vertically scanning sensors could preferably be used together to be able to detect both horizontal and vertical motion. Additionally, variations to the disclosed embodiments can be understood and effected by the skilled person in practicing the claimed disclosure, from a study of the drawings, the disclosure, and the appended claims.

Claims

CLAIMS1. A motion detection method (100) for determining a motion of an object within one image comprising a plurality of pixels organized in a matrix comprising a number of rows (ri,r2,... n), and a number of columns (ci,C2,...,cm); wherein the method (100) comprises the following steps: capturing (101) the image by obtaining (102) a pixel-value from each pixel of the image in a pre-determined sequential order such that each pixel-value is obtained at a different time than pixel-values of neighboring pixels; analyzing (104) each pixel-value by comparing said pixel-value with a pixel-value of at least one neighboring pixel; and determining (106), based on the analysis (104), if the object within said image is moving.

2. The method according to claim 1, wherein the step of obtaining a pixel-value from each pixels comprises obtaining (102a) a pixel-value from each pixel of a first row (ri) starting at a pixel located at a first column (ci) and moving in a first direction towards the end column (cm) of the first row (ri); and obtaining (102b) a pixel-value from each pixel of a second row (r2) starting at a pixel located at the end column (cm) and moving in a second direction towards the first column (ci) of the second row (r2).

3. The method according to claim 2, wherein the second direction is opposite to the first direction.

4. The method according to any preceding claims, wherein the first row (ri) and the second row (r2) are adjacent to each other.

5. The method according to any preceding claims, wherein a neighboring pixel is a pixel for which its value has been obtained nearest before in time and / or nearest after in time with regard to the time of obtaining of the pixel value of a regarded pixel, according to said predetermined sequential order.

6. The method according to any preceding claims, wherein the analyzing (104) utilizing Fourier transform.

7. A system (300) for determining motion within a scene comprising one image defined by a plurality of pixels organized in a matrix comprising a number of rows (ri,r2,...,rn), and a number of columns (ci,C2,...,cm), wherein the system (300) comprises; a camera system (302) configured to capture the image; an analyzing device (304) configured analyze each pixel-value in the image; a determining device (306) configured to determine if an object within said scene is moving; and a control unit (308) configured to execute the method according to any of claims 1-6.

Images