Optical flow recognition method and readable storage medium

Abstract

The present invention provides an optical flow recognition method and a readable storage medium, the optical flow recognition method comprising: obtaining a first frame image and a second frame image, moving the second frame image according to a plurality of different displacement amounts, and comparing the second frame image with the first frame image respectively to obtain a plurality of difference amounts corresponding to the plurality of displacement amounts; outputting the displacement amount corresponding to the smallest difference amount as a candidate displacement; judging whether the candidate displacement output is accurate according to a preset threshold; if the candidate displacement output is judged to be accurate, taking the current candidate displacement output as a displacement output result, and updating the historical displacement mean based on the displacement output result; if the candidate displacement output is judged to be inaccurate, taking the historical displacement mean as the displacement output result. With such a configuration, the pixels of the image are effectively utilized, and it is not easily affected by noise, thereby improving the matching accuracy of the image and thus improving the accuracy of optical flow recognition.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an optical flow recognition method and a readable storage medium. Background Art

[0002] The optical flow algorithm uses the changes in the time domain of pixels in an image sequence to find the position movement relationship between adjacent frames, thereby calculating the motion information of the object. The application scope of the optical flow algorithm includes mouse, knob, encoder, robot positioning, etc.

[0003] The existing optical flow algorithm is greatly affected by noise and does not make sufficient use of the original image information, resulting in low matching accuracy. Summary of the invention

[0004] The object of the present invention is to provide an optical flow recognition method and a readable storage medium to solve the problem that the existing optical flow algorithm has low matching accuracy.

[0005] In order to solve the above technical problems, the present invention provides an optical flow recognition method, which comprises: obtaining a first frame image and a second frame image, moving the second frame image according to a plurality of different displacement amounts, and comparing the second frame image with the first frame image respectively, to obtain a plurality of difference amounts corresponding to the plurality of displacement amounts;

[0006] Outputting the displacement amount corresponding to the smallest difference amount as a candidate displacement;

[0007] According to a preset threshold, determining whether the candidate displacement output is accurate;

[0008] If it is determined that the candidate displacement output is accurate, the current candidate displacement output is used as the displacement output result, and the historical displacement mean is updated based on the displacement output result;

[0009] If it is determined that the candidate displacement output is inaccurate, the historical displacement mean is used as the displacement output result.

[0010] Optionally, the overlapping pixels in the first frame image and the second frame image are a group of corresponding pixels, and the difference amount is the ratio of the sum of the absolute values ​​of the characteristic differences of all the corresponding pixels to the number of overlapping pixels.

[0011] Optionally, the overlapping pixels in the first frame image and the second frame image are a group of corresponding pixels, and the difference includes at least one of the image average brightness difference of all the corresponding pixels, the image standard deviation of all the corresponding pixels, and the image correlation coefficient of all the corresponding pixels.

[0012] Optionally, the step of judging whether the candidate displacement output is accurate according to a preset threshold value includes:

[0013] Obtaining an average value of the difference; the average value of the difference is an average value of all the difference, or the average value of the difference corresponding to adjacent displacements of the displacement corresponding to the smallest difference;

[0014] Obtaining the ratio of the minimum difference to the average value of the difference;

[0015] If the ratio is not greater than the threshold, it is determined that the candidate displacement output is accurate; if the ratio is greater than the threshold, it is determined that the candidate displacement output is inaccurate.

[0016] Optionally, the step of updating the historical displacement mean based on the displacement output result includes:

[0017] Setting a first weight coefficient and a second weight coefficient, wherein the first weight coefficient is greater than the second weight coefficient, and the sum of the first weight coefficient and the second weight coefficient is 1;

[0018] Obtaining a first product of an original historical displacement mean and the first weight coefficient;

[0019] Obtaining a second product of the displacement output result and the second weight coefficient;

[0020] The sum of the first product and the second product is used as an updated historical displacement mean.

[0021] Optionally, after obtaining the displacement output result, the optical flow identification method further includes:

[0022] When it is determined that the candidate displacement output is inaccurate and / or the displacement output result is not zero, using the current second frame image as the first frame image in a subsequent comparison process;

[0023] When it is determined that the candidate displacement output is accurate and the displacement output result is zero, the current first frame image is used as the first frame image in a subsequent comparison process.

[0024] Optionally, the movement range of the second frame image in one direction does not exceed half of the size of the first frame image in the direction.

[0025] Optionally, the second frame image is moved in units of pixels; and the displacement includes zero.

[0026] Optionally, after obtaining a plurality of difference amounts corresponding to the plurality of displacement amounts, the optical flow identification method further includes:

[0027] Constructing a difference distribution matrix according to the difference amounts corresponding to the plurality of displacement amounts;

[0028] The difference amount corresponding to the displacement amount being zero is located at the center of the difference distribution matrix; the difference amount corresponding to the displacement amount corresponding to the movement of the second frame image in one direction is located at the corresponding position of the difference distribution matrix in the direction.

[0029] In order to solve the above technical problem, the present invention further provides a readable storage medium on which a program is stored. When the program is executed, the steps of the optical flow recognition method as described above are implemented.

[0030] In summary, in the optical flow recognition method and readable storage medium provided by the present invention, the optical flow recognition method includes: acquiring a first frame image and a second frame image, moving the second frame image according to multiple different displacement amounts, and then comparing them with the first frame image respectively to obtain multiple difference amounts corresponding to the multiple displacement amounts; outputting the displacement amount corresponding to the smallest difference amount as a candidate displacement; judging whether the candidate displacement output is accurate according to a preset threshold; if the candidate displacement output is judged to be accurate, using the current candidate displacement output as a displacement output result, and updating the historical displacement mean based on the displacement output result; if the candidate displacement output is judged to be inaccurate, using the historical displacement mean as the displacement output result.

[0031] With such a configuration, a candidate displacement output is selected based on the difference between the second frame image and the first frame image after it is moved relative to the first frame image, and then the final displacement output result is determined based on the judgment result of whether the candidate displacement output is accurate and combined with the historical displacement mean, and the historical displacement mean is updated. This effectively utilizes the pixels of the image and is not easily affected by noise, thereby improving the matching accuracy of the image and thus improving the accuracy of optical flow recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Those skilled in the art will appreciate that the drawings are provided for a better understanding of the present invention, but do not constitute any limitation on the scope of the present invention.

[0033] Figure 1 4 is a flow chart of an optical flow recognition method according to an embodiment of the present invention.

[0034] Figure 2 is a schematic diagram of a first frame image according to an embodiment of the present invention.

[0035] Figure 3 is a schematic diagram of a second frame image according to an embodiment of the present invention.

[0036] Figure 4 It is a schematic diagram of comparison between the second frame image and the first frame image when the displacement amount of the second frame image is zero according to an embodiment of the present invention.

[0037] Figure 53 is a schematic diagram of comparison between the second frame image and the first frame image when the displacement amount of the second frame image is (dx=1, dy=0) according to an embodiment of the present invention.

[0038] Figure 6 Schematic diagram of a difference distribution matrix according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, advantages and features of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the drawings are all in a very simplified form and are not drawn to scale, and are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention. In addition, the structure shown in the drawings is often a part of the actual structure. In particular, the emphasis of each drawing is different, and sometimes different scales are used.

[0040] As used in the present invention, the singular forms "one", "an", "one" and "the" include plural objects, the term "or" is generally used to include the meaning of "and / or", the term "several" is generally used to include the meaning of "at least one", and the term "at least two" is generally used to include the meaning of "two or more". In addition, the terms "first", "second" and "third" are used only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first", "second" and "third" may explicitly or implicitly include one or at least two of the features, and "one end" and "the other end" and "the proximal end" and "the distal end" generally refer to two corresponding parts, which include not only the endpoints. In addition, directional terms such as above, below, up, down, upward, downward, left, right, etc. are used relative to the exemplary embodiments as they are shown in the figures, with the upward or upper direction toward the top of the corresponding figure, and the downward or lower direction toward the bottom of the corresponding figure.

[0041] The purpose of the present invention is to provide an optical flow recognition method and a readable storage medium to solve the problem of low matching accuracy of existing optical flow algorithms. The optical flow recognition method provided in this embodiment is mainly used to identify the position movement relationship between the front and rear image frames in the image. In some application scenarios, the optical flow recognition method provided in this embodiment can be performed directly inside the chip, such as an optical mouse chip, a knob encoder chip, etc. Due to limited chip resources, the memory occupancy and computational complexity of the algorithm cannot be too high, so a simpler algorithm needs to be adopted. Of course, in other application scenarios, the optical flow recognition method provided in this embodiment can also be used in a processing unit, such as optical flow positioning of a drone. The following is described with reference to the accompanying drawings.

[0042] Please refer to Figure 1, an embodiment of the present invention provides an optical flow recognition method, which includes:

[0043] Step S1: acquiring a first frame image and a second frame image, moving the second frame image by a plurality of different displacement amounts, and then comparing the second frame image with the first frame image to obtain a plurality of difference amounts corresponding to the plurality of displacement amounts;

[0044] Step S2: outputting the displacement corresponding to the smallest difference as a candidate displacement;

[0045] Step S3: judging whether the candidate displacement output is accurate according to a preset threshold;

[0046] Step S3A: if it is determined that the candidate displacement output is accurate, the current candidate displacement output is used as the displacement output result, and the historical displacement mean is updated based on the displacement output result;

[0047] Step S3B: If it is determined that the candidate displacement output is inaccurate, the historical displacement mean is used as the displacement output result.

[0048] In step S1, the first frame image and the second frame image can be two adjacent frames arranged in chronological order, but are not limited to being adjacent frames, and can also be interval frames arranged in chronological order. The first frame image is used as the basic frame of the entire recognition process. In particular, the first frame image here is not limited to the first frame of the image, but can be any frame image selected in the image, and the second frame image is arranged after the first frame image in time. The second frame image is moved with the basic frame as a reference object and compared with the basic frame to obtain the difference. The difference is a quantitative expression of the difference between the second frame image and the first frame image, which objectively reflects the difference between the second frame image and the first frame image. The smaller the difference, the more similar the second frame image is to the first frame image, or the more overlapping or identical elements the second frame image contains with the first frame image.

[0049] Please refer to Figures 2 to 5 , take an image containing 5*5 pixels as an example, Figure 2 The first frame image is shown. Figure 3 The second frame image is shown. Figures 2 to 5 The horizontal direction is x and the vertical direction is y. Optionally, the second frame image moves in units of pixels, and the displacement includes zero. That is, the second frame image may not move relative to the first frame image, and the displacement is zero, that is, (dx=0, dy=0). At this time, all pixels in the second frame image and the first frame image overlap. At this time, the comparison between the second frame image and the first frame image is as follows: Figure 4As shown. At this time, the difference between the second frame image and the first frame image when the displacement is zero can be obtained. It should be noted that the image of the embodiment of the present invention is illustrated by taking orthogonally arranged pixels as an example. In some other embodiments, the pixels contained in the image are not limited to being orthogonally arranged in the x-direction and y-direction. The pixels can also be arranged in a honeycomb shape. In addition, in some application scenarios, the image can also only move along a one-dimensional linear direction, and the corresponding image displacement can only count the coordinates in one direction. For example, in an application for identifying mouse presses, it may only be necessary to identify the displacement in the pressing direction.

[0050] Please refer to Figure 5 , which shows the comparison of the second frame image after it is moved one pixel to the right relative to the first frame image. That is, the displacement at this time is (dx=1, dy=0). At this time, the 4*5 pixels in the middle of the second frame image and the first frame image overlap. At this time, the difference between the second frame image and the first frame image when the displacement is (dx=1, dy=0) can be obtained, that is, the difference in the overlapping area. It can be seen that after the translation, the second frame image has more overlapping elements with the first frame image, and the difference is smaller than the difference when the displacement is zero. Similarly, the difference corresponding to the second frame image after moving by other displacements (that is, when dx and dy are at other values) can be obtained, thereby obtaining a set of multiple difference amounts corresponding to multiple displacements.

[0051] In an alternative exemplary embodiment, the overlapping pixels in the first frame image and the second frame image are a group of corresponding pixels, and the difference amount is the ratio of the sum of the absolute values ​​of the characteristic differences of all the corresponding pixels to the number of overlapping pixels, that is, the average SAD. It can be understood that after the second frame image moves relative to the first frame image, some pixels on the edge will not overlap and correspond. For example, referring to Figure 5 , the pixels of the second frame image overlapping with the first frame image are the 4*5 area in the middle, that is, there are 20 groups of corresponding pixels. The sum of the absolute values ​​of the characteristic differences of these corresponding pixels is SAD. The characteristic difference here can be, for example, brightness difference or hue difference. Taking brightness difference as an example, the calculation formula of SAD is as follows:

[0052]

[0053] Among them, P1 ij is the brightness value of the pixel with the corresponding pixel coordinates (referring to the pixel coordinates of the overlapping area) (i, j) in the first frame image, P2 ij is the brightness value of the pixel with the corresponding pixel coordinate (i, j) in the second frame image. m and n are the number of pixels in the x and y directions of the overlapping area. Figure 5In the example shown, m=4, n=5. It can be seen that SAD is the sum of the absolute values ​​of the brightness differences of all 20 groups of corresponding pixels. It can be understood that when the second frame image moves by different displacement amounts, the range of the overlapping area will also be different, and the number of corresponding pixels it contains will also be different. Therefore, the SAD obtained at this time cannot be directly compared, so the SAD is further divided by the number of corresponding pixels to obtain the average SAD. The average SAD can be directly used to evaluate the similarity between the overlapping areas of the second frame image and the first frame image. The smaller the average SAD value, the more similar the compared images are.

[0054] It should be noted that the average SAD is not the only option for the difference amount. In some other embodiments, the difference amount may also include at least one of the image average brightness difference of all corresponding pixels, the image standard deviation of all corresponding pixels, and the image correlation coefficient (CORR) of all corresponding pixels. Figure 5 In the example shown, taking the average brightness difference of the image as an example, the average brightness of the 20 pixels in the overlapping area corresponding to the first frame image and the average brightness of the 20 pixels in the overlapping area corresponding to the second frame image can be calculated respectively, and then the average brightness difference between the two can be calculated as the difference amount. The image correlation coefficient (CORR) is a reference coefficient that characterizes whether the images are matched and related. When the image correlation coefficient (CORR) is close to 1, it indicates that the two images are very matched. In practical applications, 1 minus the image correlation coefficient (CORR) can be used as the difference amount. Optionally, the difference amount can also be a combination of the above-mentioned multiple optional items. For example, certain weights can be set for the above-mentioned items respectively, and finally the weighted sum is applied. The present invention is not limited to this.

[0055] Optionally, the moving range of the second frame image in one direction does not exceed half of the size of the first frame image in the direction. As the moving distance of the second frame image increases, the number of overlapping corresponding pixels will also decrease, resulting in an increase in the influence of noise on the calculation result. Therefore, it is necessary to limit the moving range of the second frame image. The moving range can be set to half of the number of pixels of the entire image. For example, for an image of 18*18 pixels, the moving ranges in the x and y directions can be -9 to +9 respectively.

[0056] Optional, please refer to Figure 6 After obtaining a plurality of difference amounts corresponding to the plurality of displacement amounts, the optical flow identification method further comprises:

[0057] Step S11: constructing a difference distribution matrix according to the difference amounts corresponding to the plurality of displacement amounts;

[0058] Step S12: wherein the difference corresponding to the displacement being zero is located at the center of the difference distribution matrix; the difference corresponding to the displacement corresponding to the movement of the second frame image in one direction is located at the corresponding position of the difference distribution matrix in the direction.

[0059] In some embodiments, after the difference amounts corresponding to the multiple displacement amounts are obtained in step S1, the difference amounts corresponding to the different displacement amounts may be recorded in a difference distribution matrix. Figure 6 The difference distribution matrix shown is based on Figures 2 to 5 For example, taking the difference as the average SAD, the 78 in the middle position indicates that the displacement of the second frame image is zero (corresponding to Figure 4 ), that is, the average SAD when (dx=0, dy=0) is 78. The right column corresponds to the average SAD when dx=1 (where Figure 5 The corresponding average SAD is 45), the left column corresponds to the average SAD of dx=-1, the upper row corresponds to the average SAD of dy=-1, and the lower row corresponds to the average SAD of dy=1. It is understandable that Figure 6 Only the difference distribution matrix constructed by the 9 difference amounts corresponding to the 9 displacement amounts when the second frame image is one pixel above, below, left, and right relative to the first frame image and the displacement amount is zero is shown. In practice, the difference distribution matrix is ​​not limited to only 9 difference amounts, but can be constructed according to the displacement amount actually moved by the second frame image. Furthermore, in some application scenarios, in order to save storage, the difference distribution matrix may not really need to be stored, but only the minimum difference amount and the corresponding displacement amount in the calculation process are recorded.

[0060] In step S2, the displacement corresponding to the smallest difference is output as a candidate displacement; it can be understood that when the difference is the smallest, it means that the elements overlapping the second frame image and the first frame image are relatively the largest. At this time, the displacement of the second frame image may be the exact position movement relationship between the second frame image and the first frame image, and of course, it may also be a misjudgment caused by interference factors (such as noise or changes in the object itself between the second frame image and the first frame image). Therefore, at this time, the displacement corresponding to the smallest difference cannot be directly used as a determined displacement output result, and it is necessary to judge whether the candidate displacement output is accurate through step S3.

[0061] It can be understood that when the difference of the candidate displacement output is significantly smaller than the difference under other displacements, the candidate displacement output can be considered accurate. On the contrary, if the difference of the candidate displacement output is relatively small, but the difference between it and the difference under other displacements is not big, the candidate displacement output is considered inaccurate. In the judgment of step S3, the preset threshold can be set according to the feature actually corresponding to the difference.

[0062] In an alternative example, step S3 includes:

[0063] Step S31: obtaining an average value of the difference; the average value of the difference is the average value of all the difference, or the average value of the difference corresponding to the adjacent displacements corresponding to the smallest difference;

[0064] Step S32: Obtaining the ratio of the minimum difference to the average difference;

[0065] Step S33: if the ratio is not greater than the threshold, it is determined that the candidate displacement output is accurate; if the ratio is greater than the threshold, it is determined that the candidate displacement output is inaccurate.

[0066] Step S3 can determine whether the difference between the candidate displacement output and the difference between other displacement outputs is significant by setting certain rules. Figure 6 In the difference distribution matrix shown, the smallest difference is 2 in the lower right corner, which is significantly smaller than other differences. Therefore, it can be considered that the candidate displacement output is accurate at this time.

[0067] In an example, the average value of all differences can be taken. Optionally, if the difference distribution matrix is ​​stored, all differences in the difference distribution matrix can be directly selected to calculate their average values. Taking the difference as the average SAD as an example, the average value of all differences is recorded as OSAD. Then, the smallest average SAD is divided by OSAD. The smaller the ratio obtained, the higher the probability that the displacement corresponding to the smallest average SAD is accurate.

[0068] In another example, in order to save storage, the difference distribution matrix is ​​not stored. In this case, after determining the displacement corresponding to the minimum difference, the displacement can be used as a reference to calculate the differences corresponding to several adjacent displacements, and calculate their average value. The neighborhood here can be determined based on the image size and pixel count of the actual image. In the simplest example, taking the neighborhood range as 1 pixel, when the displacement corresponding to the minimum average SAD is (dx=2, dy=3), the average SAD corresponding to several displacements of dx=1, dx=3, dy=2, dy=4 is recalculated, and then the OSAD is calculated.

[0069] Step S3A and step S3B are two judgment branches of step S3. A new parameter is introduced here: the historical displacement mean. The historical displacement mean is a comprehensive evaluation value of the relatively reliable displacement amount based on the previous image frames in the image. It can be understood that when the steps of the optical flow recognition method of this embodiment are executed for the first time, the historical displacement mean does not exist, and it can be initialized to zero at this time.

[0070] In step S3A, if the candidate displacement output is determined to be accurate, the current candidate displacement output is directly used as the displacement output result. The historical displacement mean is further updated based on the displacement output result (ie, the current candidate displacement output) so as to be used as a reference for the next round of recognition of subsequent image frames.

[0071] In step S3B, if the candidate displacement output is judged to be inaccurate, the historical displacement mean is used as the displacement output result, that is, the candidate displacement output is discarded at this time, and the historical displacement mean stored in the previous step is selected as the displacement output result. It can be understood that the historical displacement mean will not be updated at this time.

[0072] The optical flow recognition method of this embodiment judges whether the candidate displacement output is accurate, and then performs different operations according to the judgment branch. When the candidate displacement output is judged to be inaccurate, the historical displacement mean is used as the displacement output result, which greatly improves the tolerance rate for interference and misjudgment, is not easily affected by noise, improves the matching accuracy of the image, and thus improves the accuracy of optical flow recognition.

[0073] Optionally, the step of updating the historical displacement mean based on the displacement output result in step S3A includes:

[0074] Step S3A1: setting a first weight coefficient k1 and a second weight coefficient k2, wherein the first weight coefficient k1 is greater than the second weight coefficient k2, and the sum of the first weight coefficient k1 and the second weight coefficient k2 is 1;

[0075] Step S3A2: obtaining a first product of the original historical displacement mean and the first weight coefficient k1;

[0076] Step S3A3: obtaining a second product of the displacement output result and the second weight coefficient k2;

[0077] Step S3A4: taking the sum of the first product and the second product as the updated historical displacement mean.

[0078] In an alternative exemplary embodiment, step S3A determines whether the candidate displacement output is accurate, and the displacement output result is set to (dx, dy), and the original historical displacement average (i.e., the historical displacement average before updating) is set to (dx_avg, dy_avg). The update of the historical displacement average is shown in the following formula:

[0079] dx_avg=k1* dx_avg+ k2* dx, dy_avg=k1* dy_avg+ k2* dy.

[0080] Generally, the movement of an object has a certain degree of continuity, and the historical displacement mean is the displacement trend of the image before the current identification step. The current accurate displacement output result is included in the historical displacement mean according to a certain weight (the second weight coefficient k2), and the displacement trend of the image can be corrected. The updated historical displacement mean can be used to guide and predict the displacement of the next frame of the image. It can be understood that in order to prevent the displacement output result obtained by a certain frame of the image from having a sudden impact on the historical displacement mean, the first weight coefficient k1 should be larger, and the second weight coefficient k2 should be smaller. In an example, the first weight coefficient k1 can be, for example, selected as 0.7~0.95, preferably 0.85, 0.9; the second weight coefficient k2 can be, for example, selected as 0.05~0.3, preferably 0.15, 0.1.

[0081] Optionally, after obtaining the displacement output result, the optical flow identification method further includes:

[0082] Step S4: when it is determined that the candidate displacement output is inaccurate and / or the displacement output result is not zero, using the current second frame image as the first frame image in a subsequent comparison process;

[0083] Step S5: when it is determined that the candidate displacement output is accurate and the displacement output result is zero, the current first frame image is used as the first frame image in the subsequent comparison process.

[0084] The settings of step S4 and step S5 can reduce the read and write operations of the register.

[0085] Based on the above optical flow recognition method, an embodiment of the present invention further provides a readable storage medium on which a program is stored. When the program is executed, the steps of the above optical flow recognition method are implemented. The readable storage medium can be integrated in a chip or a processing unit, or can be independently set, and the present invention is not limited to this.

[0086] In summary, in the optical flow recognition method and readable storage medium provided by the present invention, the optical flow recognition method includes: obtaining a first frame image and a second frame image, moving the second frame image according to multiple different displacement amounts, and then comparing them with the first frame image respectively to obtain multiple difference amounts corresponding to the multiple displacement amounts; using the displacement amount corresponding to the smallest difference amount as a candidate displacement output; judging whether the candidate displacement output is accurate according to a preset threshold; if the candidate displacement output is judged to be accurate, using the current candidate displacement output as a displacement output result, and updating the historical displacement mean based on the displacement output result; if the candidate displacement output is judged to be inaccurate, using the historical displacement mean as the displacement output result. In this configuration, the candidate displacement output is selected based on the difference amount obtained by comparing the second frame image with the first frame image after it is moved relative to the first frame image, and then the final displacement output result is determined based on the judgment result of whether the candidate displacement output is accurate and combined with the historical displacement mean, and the historical displacement mean is updated, which effectively utilizes the pixels of the image, is not easily affected by noise, improves the matching accuracy of the image, and thus improves the accuracy of optical flow recognition.

[0087] It should be noted that the above embodiments can be combined with each other. The above description is only a description of the preferred embodiments of the present invention, and does not limit the scope of the present invention. Any changes and modifications made by ordinary technicians in the field of the present invention based on the above disclosure are within the protection scope of the present invention.

Claims

1. An optical flow recognition method, characterized in that: include: Acquire a first frame image and a second frame image, wherein overlapping pixels in the first frame image and the second frame image are a group of corresponding pixels; After moving the second frame image according to a plurality of different displacement amounts, the second frame image is respectively compared with the first frame image to obtain a plurality of difference amounts corresponding to the plurality of displacement amounts; Outputting the displacement amount corresponding to the smallest difference amount as a candidate displacement; According to a preset threshold, judging whether the candidate displacement output is accurate, the steps include: obtaining an average value of differences; the average value of differences is the average value of all differences, or the average value of differences is the average value of differences corresponding to adjacent displacements corresponding to the smallest difference; obtaining a ratio of the smallest difference to the average value of differences; if the ratio is not greater than the threshold, judging that the candidate displacement output is accurate; if the ratio is greater than the threshold, judging that the candidate displacement output is inaccurate; If it is determined that the candidate displacement output is accurate, the current candidate displacement output is used as the displacement output result, and the historical displacement mean is updated based on the displacement output result; If it is determined that the candidate displacement output is inaccurate, the historical displacement mean is used as the displacement output result.

2. The optical flow identification method according to claim 1, characterized in that: The difference amount is the ratio of the sum of the absolute values ​​of the characteristic differences of all the corresponding pixels to the number of overlapping pixels.

3. The optical flow recognition method according to claim 1, characterized in that: The overlapping pixels in the first frame image and the second frame image are a group of corresponding pixels, and the difference includes at least one of the image average brightness difference of all the corresponding pixels, the image standard deviation of all the corresponding pixels, and the image correlation coefficient of all the corresponding pixels.

4. The optical flow identification method according to claim 1, characterized in that: The step of updating the historical displacement mean based on the displacement output result comprises: Setting a first weight coefficient and a second weight coefficient, wherein the first weight coefficient is greater than the second weight coefficient, and the sum of the first weight coefficient and the second weight coefficient is 1; Obtaining a first product of an original historical displacement mean and the first weight coefficient; Obtaining a second product of the displacement output result and the second weight coefficient; The sum of the first product and the second product is used as an updated historical displacement mean.

5. The optical flow identification method according to claim 1, characterized in that: After obtaining the displacement output result, the optical flow identification method further includes: When it is determined that the candidate displacement output is inaccurate and / or the displacement output result is not zero, using the current second frame image as the first frame image in a subsequent comparison process; When it is determined that the candidate displacement output is accurate and the displacement output result is zero, the current first frame image is used as the first frame image in a subsequent comparison process.

6. The optical flow identification method according to claim 1, characterized in that: The moving range of the second frame image in one direction does not exceed half of the size of the first frame image in the direction.

7. The optical flow identification method according to claim 1, characterized in that: The second frame image is moved in units of pixels; and the displacement includes zero.

8. The optical flow identification method according to claim 1, characterized in that: After obtaining a plurality of difference amounts corresponding to the plurality of displacement amounts, the optical flow recognition method further comprises: Constructing a difference distribution matrix according to the difference amounts corresponding to the plurality of displacement amounts; The difference amount corresponding to the displacement amount being zero is located at the center of the difference distribution matrix; the difference amount corresponding to the displacement amount corresponding to the movement of the second frame image in one direction is located at the corresponding position of the difference distribution matrix in the direction.

9. A readable storage medium having a program stored thereon, characterized in that: When the program is executed, the steps of the optical flow recognition method according to any one of claims 1 to 8 are implemented.

Images