Optical flow calculation method, device, electronic device and computer-readable medium

Abstract

The present application provides an optical flow calculation method, apparatus, electronic device, and computer-readable medium. The method includes: acquiring a first image and a second image; acquiring multiple first corner points of the first image and multiple second corner points of the second image, each second corner point corresponding to each first corner point; selecting a pixel matrix centered on the first corner point; and determining whether the second corner point corresponding to the first corner point is a valid corner point based on the number of corner points within the pixel matrix of the first corner point; if the second corner point that matches the first corner point is a valid corner point, obtaining an optical flow value for the second corner point based on the optical flow starting point of the second corner point and the second corner point. The present application calculates optical flow based on valid corner points, improving the efficiency of optical flow calculation while ensuring the accuracy of optical flow calculation.

Description

Technical Field

[0001] The present application relates to the field of logistics technology, and specifically to an optical flow calculation method, device, and electronic device computer-readable medium. Background Art

[0002] Optical flow is a comprehensive representation of the geometric and radiometric changes in a dynamic image. Optical flow studies use the temporal variations and correlations of pixel intensity data within an image sequence to determine the "motion" of individual pixel positions. Specifically, it studies the relationship between temporal variations in image grayscale and the structure and motion of objects in the scene. Generally, optical flow is caused by relative motion resulting from camera motion, object motion within the scene, or both.

[0003] Currently, optical flow calculation is implemented through dynamic vision sensors, which obtain two consecutive event slices within a fixed time slice range. By traversing each event point in the event slice of the next frame, a diamond search algorithm is used to find a potential matching point set, and then the optical flow value is calculated. In this way, when traversing the event points to calculate the optical flow, a large number of event points are actually redundant, which greatly reduces the efficiency of optical flow calculation. Summary of the Invention

[0004] The present application provides an optical flow calculation method, device, and computer-readable medium for electronic devices, aiming to solve the existing technical problem that when traversing event points to calculate optical flow, a large number of event points are actually redundant, resulting in a significant reduction in the efficiency of optical flow calculation.

[0005] In one aspect, the present application provides an optical flow calculation method, the optical flow calculation method comprising the following steps: acquiring a first image and a second image;

[0006] Acquire multiple first corner points of the first image, acquire multiple second corner points of the second image, each second corner point corresponds to each first corner point, select a pixel matrix centered on the first corner point, and determine whether the second corner point corresponding to the first corner point is a valid corner point based on the number of corner points in the pixel matrix of the first corner point;

[0007] If the second corner point that matches the first corner point is a valid corner point, an optical flow value of the second corner point is obtained according to the second corner point and an optical flow starting point of the second corner point.

[0008] Preferably, the determining whether the second corner point corresponding to the first corner point is a valid corner point according to the number of corner points in the pixel matrix of the first corner point includes:

[0009] Counting the number of corner points in the pixel matrix of the first corner point;

[0010] If the number of the corner points is greater than a preset number threshold, the second corner point matching the first corner point is determined to be a valid corner point.

[0011] Preferably, obtaining the optical flow starting point of the second corner point includes:

[0012] Obtaining an optical flow matching point set of the second corner point in the first image by using a preset search algorithm;

[0013] An optical flow starting point of the second corner point is selected from the optical flow matching point set.

[0014] Preferably, the selecting the optical flow starting point of the second corner point from the optical flow matching point set includes:

[0015] Constructing a first pixel matrix with the second corner point as the center; constructing a second pixel matrix with each pixel point in the optical flow matching point set as the center;

[0016] Calculating a plurality of SAD values ​​between the first pixel matrix and all the second pixel matrices;

[0017] Compare the multiple SAD values ​​and set the pixel point corresponding to the minimum SAD value as the optical flow starting point of the second corner point.

[0018] Preferably, obtaining the optical flow value of the second corner point according to the second corner point and the optical flow starting point of the second corner point includes:

[0019] Extracting the coordinates of the second corner point and the coordinates of the optical flow starting point;

[0020] The optical flow value of the second corner point is calculated according to the coordinates of the second corner point and the coordinates of the optical flow starting point.

[0021] Preferably, after obtaining the optical flow value of the second corner point according to the second corner point and the optical flow starting point of the second corner point, the method includes:

[0022] Calculate the optical flow standard deviation and the optical flow average according to the optical flow values ​​of all the second corner points;

[0023] The number of frames of the first image and the number of frames of the second image are adjusted respectively according to the optical flow standard deviation and the optical flow average value.

[0024] Preferably, adjusting the number of frames of the first image and the number of frames of the second image respectively according to the optical flow standard deviation and the optical flow average value includes:

[0025] If the optical flow standard deviation is less than a preset standard deviation threshold and the optical flow average is less than a preset optical flow threshold, simultaneously increasing the number of frames of the first image and the number of frames of the second image;

[0026] If the optical flow standard deviation is smaller than a preset standard deviation threshold or the optical flow average value is larger than a preset optical flow threshold, the number of frames of the first image and the number of frames of the second image are reduced simultaneously.

[0027] In another aspect, the present application provides an optical flow calculation device, comprising:

[0028] An image synthesis module, configured to obtain a first image and a second image;

[0029] an image processing module, configured to obtain a plurality of first corner points of the first image, obtain a plurality of second corner points of the second image, each second corner point corresponding to each first corner point, select a pixel matrix centered around the first corner point, and determine whether the second corner point corresponding to the first corner point is a valid corner point based on the number of corner points in the pixel matrix of the first corner point;

[0030] A calculation module is configured to obtain an optical flow value of the second corner point according to the second corner point and an optical flow starting point of the second corner point if the second corner point matching the first corner point is a valid corner point.

[0031] On the other hand, the present application further provides an electronic device, comprising:

[0032] one or more processors;

[0033] Memory; and

[0034] One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the steps in the optical flow calculation method.

[0035] On the other hand, the present application also provides a computer-readable medium having a computer program stored thereon, and the computer program is loaded by a processor to execute the steps in the optical flow calculation method.

[0036] The present application obtains a first image and a second image; obtains multiple first corner points of the first image, obtains multiple second corner points of the second image, each second corner point corresponds to each first corner point, selects a pixel matrix with the first corner point as the center, and determines whether the second corner point corresponding to the first corner point is a valid corner point based on the number of corner points in the pixel matrix of the first corner point; if the second corner point matching the first corner point is a valid corner point, obtains the optical flow value of the second corner point based on the second corner point and the optical flow starting point of the second corner point; in the embodiment of the present application, the first corner point in the first image is used to assist the second image to determine the validity of the second corner point, so that the second image can remove redundant event points and noise points to obtain valid corner points, and performs optical flow calculation based on the motion information of the valid corner points, thereby effectively improving the calculation efficiency of the optical flow, and effectively removing the erroneous optical flow generated by noise, and the accuracy is also greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 This is a flowchart of an embodiment of the optical flow calculation method in the embodiment of the present application;

[0039] Figure 2 This is a schematic diagram of an embodiment of a scene in which the first image and the second image are determined by the optical flow calculation method in an embodiment of the present application;

[0040] Figure 3 This is a schematic diagram of an embodiment of a scene for determining effective corner points using the optical flow calculation method according to an embodiment of the present application;

[0041] Figure 4 This is a schematic diagram of an embodiment scenario of the diamond search algorithm in the optical flow calculation method in the embodiment of the present application;

[0042] Figure 5 Schematic diagram of a pixel matrix for optical flow calculation in the optical flow calculation method provided in an embodiment of the present application;

[0043] Figure 6 Schematic diagram of the optical flow calculation process in the optical flow calculation method provided in the embodiment of the present application;

[0044] Figure 7 1 is a flow chart of adjusting the number of frames of the first image and the number of frames of the second image according to the optical flow value provided in an embodiment of the present application;

[0045] Figure 8 It is a schematic diagram of the structure of an embodiment of an electronic device provided in the embodiments of the present application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of the present invention.

[0047] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the said features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0048] In this application, the word "exemplary" is used to mean "serving as an example, illustration, or illustration." Any embodiment described in this application as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is given to enable any person skilled in the art to make and use the invention. In the following description, details are listed for the purpose of explanation. It should be understood that one of ordinary skill in the art will recognize that the invention can be practiced without these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

[0049] Example 1

[0050] The present application provides an optical flow calculation method, such as Figure 1-7 As shown, the optical flow calculation method includes steps 100-300:

[0051] Step 100, acquiring a first image and a second image;

[0052] Specifically, multiple images taken in the first time period (0-t) are merged into the event point cumulative frame frame1, that is, the first image, and multiple images taken in the second time period (t-2t) are merged into the event point cumulative frame frame2, that is, the second image. Time t is the unit length, that is, time t can be set to 1s, among which the time interval of the 0-t time period can be adjusted, and the time interval within the t-2t time period can also be adjusted.

[0053] It is also worth noting that in this embodiment, the number of images captured per unit time is fixed, and the specific number of images captured can be selected according to actual needs. In this embodiment, it is set that 60 images are captured in 1 second, and t is set to 5 seconds. Then, 300 images are captured from 0 to 5 seconds, and 300 images are captured from 5 to 10 seconds. The 300 images captured from 0 to 5 seconds are merged into the event point cumulative frame frame1, that is, the first image, and the 300 images captured from 5 to 10 seconds are merged into the event point cumulative frame frame2, that is, the second image.

[0054] Step 200: Acquire multiple first corner points of the first image, acquire multiple second corner points of the second image, each second corner point corresponds to each first corner point, select a pixel matrix centered on the first corner point, and determine whether the second corner point corresponding to the first corner point is a valid corner point based on the number of corner points in the pixel matrix of the first corner point.

[0055] It should be noted that an xy coordinate system is established for the first image and the second image respectively, wherein the coordinate origin of the xy coordinate system of the first image is the same as the coordinate origin of the coordinate system of the second image, and the coordinates of the first corner point in the xy coordinate system of the first image are: c1(x1, y1), and the coordinates of the second corner point in the xy coordinate system of the second image are: c2(x2, y2), wherein the coordinates of c1 and c2 are the same, then the first corner point and the second corner point correspond to each other.

[0056] In addition, specifically, a pixel matrix is ​​selected in the first image with the first corner point as the center, where the size of the pixel matrix can be flexibly set. For example, the pixel matrix is ​​5*5, and the number of corner points in the pixel matrix is ​​counted. According to the number of corner points in the pixel matrix, it is determined whether the second corner point is a valid corner point.

[0057] In this embodiment, whether the second corner point is a valid corner point is first determined based on the conditions of the pixels around the first corner point. This can reduce the calculation of redundant data, improve calculation efficiency and ensure calculation accuracy.

[0058] Step 300: If the second corner point that matches the first corner point is a valid corner point, obtain an optical flow value of the second corner point according to the second corner point and an optical flow starting point of the second corner point.

[0059] If the second corner point is a valid corner point, the optical flow matching point set of the valid corner point is obtained by using a preset diamond search algorithm, wherein the optical flow matching point set is the position of the potential optical flow starting point determined in the preset diamond search algorithm, and any event point in the first image belongs to the optical flow matching point set; in this application, the optical flow matching point set of the second corner point is obtained by using a preset diamond search algorithm.

[0060] The present application obtains a first image and a second image; obtains multiple first corner points of the first image, obtains multiple second corner points of the second image, each second corner point corresponds to each first corner point, selects a pixel matrix with the first corner point as the center, and determines whether the second corner point corresponding to the first corner point is a valid corner point based on the number of corner points in the pixel matrix of the first corner point; if the second corner point matching the first corner point is a valid corner point, obtains the optical flow value of the second corner point based on the second corner point and the optical flow starting point of the second corner point; in the embodiment of the present application, the first corner point in the first image is used to assist the second image to determine the validity of the second corner point, so that the second image can remove redundant event points and noise points to obtain valid corner points, and performs optical flow calculation based on the motion information of the valid corner points, thereby effectively improving the calculation efficiency of the optical flow, and effectively removing the erroneous optical flow generated by noise, and the accuracy is also greatly improved.

[0061] Preferably, in step 300, if the second corner point matched with the first corner point is a valid corner point, an optical flow value of the second corner point is obtained according to the second corner point and an optical flow starting point of the second corner point.

[0062] include:

[0063] Step 310: Count the number of corner points in the pixel matrix of the first corner point;

[0064] Step 320: If the number of the corner points is greater than a preset number threshold, determine that the second corner point matching the first corner point is a valid corner point.

[0065] Specifically, with the first corner point as the center, a 3×3 area is selected on the first image and divided into areas A and B. The number of corner points n1 and n2 in areas A and B are counted respectively. Only when n1>n and n2>n (n is a preset threshold), the second corner point is considered a valid corner point and the next step of calculation is performed. Otherwise, it is considered an invalid corner point and no calculation is performed.

[0066] Preferably, obtaining the optical flow starting point of the second corner point includes:

[0067] Step 330: Obtain an optical flow matching point set of the second corner point in the first image using a preset search algorithm;

[0068] For example, when the second corner point is a valid corner point, the area of ​​s×s is selected with the first corner point as the center, and the potential optical flow matching point set M is selected through the diamond search algorithm. n (x n ,y n ).

[0069] Step 340: Select an optical flow starting point of the second corner point from the optical flow matching point set.

[0070] Specifically, step 340, selecting the second corner optical flow starting point from the optical flow matching point set, includes:

[0071] Step 340A: construct a first pixel matrix with the second corner point as the center; construct a second pixel matrix with each pixel point in the optical flow matching point set as the center

[0072] Step 340B, calculating a plurality of SAD values ​​between the first pixel matrix and all the second pixel matrices;

[0073] Step 340C: compare the multiple SAD values, and set the pixel point corresponding to the minimum SAD value as the optical flow starting point of the second corner point.

[0074] For example, when the second corner point is a valid corner point, in the second image, a rhombus is formed with the second corner point as the center, forming 9 pixels, thereby forming a pixel matrix including 9 pixel blocks a1-a9. In the first image, a rhombus is formed with point M1 as the center, forming 9 pixels, and including 9 pixel blocks b1-b9. The SAD1 value between the first pixel matrix constructed with the second corner point as the center and the second pixel matrix constructed with point M1 as the center is:

[0075]

[0076] The same method is used to calculate the SADi value between the first pixel matrix constructed with the second corner point as the center and the pixel matrix constructed with other pixel points Mi, i=2,,,n in the optical flow matching point set as the center. By comparing the SAD1 value and the SADi value (i=2,,,n), the pixel point corresponding to the smallest value between the SAD1 value and the SADi value can be used as the flow starting point of the second corner point. For example, if the SAD1 value is the smallest between the SAD1 value and the SADi value, M1 is used as the optical flow starting point of the second corner point.

[0077] Preferably, in step 300, obtaining the optical flow value of the second corner point according to the second corner point and the optical flow starting point of the second corner point includes:

[0078] Step 300a, extracting the coordinates of the second corner point and the coordinates of the optical flow starting point;

[0079] Step 300b: Calculate the optical flow value of the second corner point based on the coordinates of the second corner point and the coordinates of the optical flow starting point.

[0080] For example, if M1 is the starting point of the optical flow of the second corner point, let the coordinates of the second corner point be c2(x2, y2), and the coordinates of M1 are: (x m1 ,y m2 ), then the optical flow value of the first corner point c1 is:

[0081]

[0082] Preferably, step 300, after obtaining the optical flow value of the second corner point according to the second corner point and the optical flow starting point of the second corner point, includes:

[0083] Step 400: Calculate the optical flow standard deviation and the optical flow average according to the optical flow values ​​of all the second corner points;

[0084] Step 500: Adjust the number of frames of the first image and the number of frames of the second image respectively according to the optical flow standard deviation and the optical flow average value.

[0085] Preferably, step 500, adjusting the number of frames of the first image and the number of frames of the second image respectively according to the optical flow standard deviation and the optical flow average value, includes:

[0086] Step 510: if the optical flow standard deviation is less than a preset standard deviation threshold and the optical flow average is less than a preset optical flow threshold, then simultaneously increase the number of frames of the first image and the number of frames of the second image;

[0087] Step 520: If the optical flow standard deviation is smaller than a preset standard deviation threshold or the optical flow average value is larger than a preset optical flow threshold, the number of frames of the first image and the number of frames of the second image are reduced simultaneously.

[0088] For example: In this embodiment, the cumulative frame frame1 of event points within the time period from 0 to t is obtained at a fixed time t, that is, the first image, and the cumulative frame frame2 of event points within the time period from t to 2t is obtained at a fixed time t, that is, the second image; when the optical flow values of all the second corner points in the second image are obtained, the mean value avg and the standard deviation stddev of the optical flow values of all the second corner points are obtained. When stddev < n3, if avg < n4, it is considered that the current optical flow is small and the time t should be increased, that is, the number of frames of the first image frame1 and the number of frames of the second image frame2 are increased. If avg > n5, it is considered that the current optical flow is large and the time t should be decreased, that is, the number of frames of the first image frame1 and the number of frames of the second image frame2 are decreased. Here, n3, n4, and n5 are all preset parameter values. [[ID=!]]

[0089] In addition, the change of the time t can be increased or decreased by 1% until n4 < avg < n5, which is considered a suitable time. Usually, in a simple few-object scenario during calculation, avg represents the motion speed of the current object. When stddev > n3, it indicates that there are large speed differences in the represented scene.

[0090] It should be noted that in this embodiment, the number of images captured per unit time is fixed, that is, the frame rate is a fixed value. Since the number of frames = frame rate * time, adjusting the number of frames of frame1 and frame2 is to adjust the number of images required to be combined into frame1 and the number of images required to be combined into frame2 respectively. For example, the initial setting value is to capture 60 images in 1 second, and t is set to 5 seconds. The 300 images captured from 0 to 5 seconds are combined into the cumulative frame of event points frame1, that is, the first image, and the 300 images captured from 5 to 10 seconds are combined into the cumulative frame of event points frame2, that is, the second image. If the number of frames of the first image and the number of frames of the second image are increased, the time t can be extended, that is, the time t is set to 6 seconds. The 360 images captured from 0 to 6 seconds are combined into the cumulative frame of event points frame1, that is, the first image, and the 360 images captured from 6 to 12 seconds are combined into the cumulative frame of event points frame2, that is, the second image.

[0091] Embodiment 2

[0092] The embodiment of the present invention further provides an optical flow calculation device, which includes:

[0093] An image synthesis module, configured to obtain a first image and a second image;

[0094] an image processing module, configured to obtain a plurality of first corner points of the first image, obtain a plurality of second corner points of the second image, each second corner point corresponding to each first corner point, select a pixel matrix centered around the first corner point, and determine whether the second corner point corresponding to the first corner point is a valid corner point based on the number of corner points in the pixel matrix of the first corner point;

[0095] A calculation module is configured to obtain an optical flow value of the second corner point according to the second corner point and an optical flow starting point of the second corner point if the second corner point matching the first corner point is a valid corner point.

[0096] The present application obtains a first image and a second image; obtains multiple first corner points of the first image, obtains multiple second corner points of the second image, each second corner point corresponds to each first corner point, selects a pixel matrix with the first corner point as the center, and determines whether the second corner point corresponding to the first corner point is a valid corner point based on the number of corner points in the pixel matrix of the first corner point; if the second corner point matching the first corner point is a valid corner point, obtains the optical flow value of the second corner point based on the second corner point and the optical flow starting point of the second corner point; in the embodiment of the present application, the first corner point in the first image is used to assist the second image to determine the validity of the second corner point, so that the second image can remove redundant event points and noise points to obtain valid corner points, and performs optical flow calculation based on the motion information of the valid corner points, thereby effectively improving the calculation efficiency of the optical flow, and effectively removing the erroneous optical flow generated by noise, and the accuracy is also greatly improved.

[0097] Example 3

[0098] An embodiment of the present invention further provides an electronic device, which is a schematic structural diagram of an embodiment of the electronic device provided in the embodiment of the present application.

[0099] An electronic device integrating any of the optical flow calculation devices provided by the embodiments of the present invention includes:

[0100] one or more processors;

[0101] Memory; and

[0102] One or more applications, wherein the one or more applications are stored in the memory and are configured to cause the processor to execute the steps of the optical flow calculation method described in any of the above optical flow calculation method embodiments.

[0103] Specifically, the electronic device may include one or more processing core processors 601, one or more computer-readable storage media 602, a power supply 603, an input unit 604, and other components. Those skilled in the art will understand that Figure 8 The electronic device structure shown in the figure does not constitute a limitation to the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0104] in:

[0105] Processor 601 is the control center of the electronic device, connecting the various parts of the entire electronic device using various interfaces and lines. By running or executing software programs and / or modules stored in memory 602 and accessing data stored in memory 602, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, processor 601 may include one or more processing cores; preferably, processor 601 may integrate an application processor and a modem processor, wherein the application processor primarily handles the performance adjustment system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 601.

[0106] The memory 602 can be used to store software programs and modules. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area, wherein the program storage area may store a performance adjustment system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.

[0107] The electronic device also includes a power supply 603 for supplying power to various components. Preferably, the power supply 603 can be logically connected to the processor 601 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 603 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0108] The electronic device may further include an input unit 604, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0109] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 601 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 602 according to the following instructions, and the processor 601 will run the application programs stored in the memory 602 to implement various functions as follows:

[0110] Acquire a first image and a second image;

[0111] Acquire multiple first corner points of the first image, acquire multiple second corner points of the second image, each second corner point corresponds to each first corner point, select a pixel matrix centered on the first corner point, and determine whether the second corner point corresponding to the first corner point is a valid corner point based on the number of corner points in the pixel matrix of the first corner point;

[0112] If the second corner point that matches the first corner point is a valid corner point, an optical flow value of the second corner point is obtained according to the second corner point and an optical flow starting point of the second corner point.

[0113] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable medium and loaded and executed by a processor.

[0114] Example 4

[0115] To this end, an embodiment of the present invention provides a computer-readable medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk, or an optical disk. A computer program is stored on the computer-readable medium, and the computer program is loaded by a processor to execute the steps of any optical flow calculation method provided in the embodiment of the present invention. For example, the computer program loaded by the processor may execute the following steps:

[0116] Acquire a first image and a second image;

[0117] Acquire multiple first corner points of the first image, acquire multiple second corner points of the second image, each second corner point corresponds to each first corner point, select a pixel matrix centered on the first corner point, and determine whether the second corner point corresponding to the first corner point is a valid corner point based on the number of corner points in the pixel matrix of the first corner point;

[0118] If the second corner point that matches the first corner point is a valid corner point, an optical flow value of the second corner point is obtained according to the second corner point and an optical flow starting point of the second corner point.

[0119] In specific implementation, the above units or structures can be implemented as independent entities, or can be arbitrarily combined to implement as the same or several entities. The specific implementation of the above units or structures can refer to the previous method embodiments and will not be repeated here.

[0120] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0121] The above is a detailed introduction to an optical flow calculation method provided in an embodiment of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for calculating optical flow, characterized in that: The optical flow calculation method includes: Acquire a first image and a second image; Acquire multiple first corner points of the first image, acquire multiple second corner points of the second image, each second corner point corresponds to each first corner point, select a pixel matrix centered on the first corner point, and determine whether the second corner point corresponding to the first corner point is a valid corner point based on the number of corner points in the pixel matrix of the first corner point; If the second corner point that matches the first corner point is a valid corner point, an optical flow value of the second corner point is obtained according to the second corner point and an optical flow starting point of the second corner point.

2. The optical flow calculation method according to claim 1, characterized in that: The determining, based on the number of corner points in the pixel matrix of the first corner point, whether the second corner point corresponding to the first corner point is a valid corner point includes: Counting the number of corner points in the pixel matrix of the first corner point; If the number of the corner points is greater than a preset number threshold, the second corner point matching the first corner point is determined to be a valid corner point.

3. The optical flow calculation method according to claim 1, characterized in that: Obtaining the optical flow starting point of the second corner point includes: Obtaining an optical flow matching point set of the second corner point in the first image by using a preset search algorithm; An optical flow starting point of the second corner point is selected from the optical flow matching point set.

4. The optical flow calculation method according to claim 3, characterized in that: The step of selecting the optical flow starting point of the second corner point from the optical flow matching point set includes: Constructing a first pixel matrix with the second corner point as the center; constructing a second pixel matrix with each pixel point in the optical flow matching point set as the center; Calculating a plurality of SAD values ​​between the first pixel matrix and all the second pixel matrices; Compare the multiple SAD values ​​and set the pixel point corresponding to the minimum SAD value as the optical flow starting point of the second corner point.

5. The optical flow calculation method according to claim 1, characterized in that: Acquiring an optical flow value of the second corner point according to the second corner point and an optical flow starting point of the second corner point includes: Extracting the coordinates of the second corner point and the coordinates of the optical flow starting point; The optical flow value of the second corner point is calculated according to the coordinates of the second corner point and the coordinates of the optical flow starting point.

6. The optical flow calculation method according to any one of claims 1 to 5, characterized in that: After obtaining the optical flow value of the second corner point according to the second corner point and the optical flow starting point of the second corner point, the method includes: Calculate the optical flow standard deviation and the optical flow average according to the optical flow values ​​of all the second corner points; The number of frames of the first image and the number of frames of the second image are adjusted respectively according to the optical flow standard deviation and the optical flow average value.

7. The optical flow calculation method according to claim 6, characterized in that: The adjusting the number of frames of the first image and the number of frames of the second image respectively according to the optical flow standard deviation and the optical flow average value includes: If the optical flow standard deviation is less than a preset standard deviation threshold and the optical flow average is less than a preset optical flow threshold, simultaneously increasing the number of frames of the first image and the number of frames of the second image; If the optical flow standard deviation is smaller than a preset standard deviation threshold or the optical flow average value is larger than a preset optical flow threshold, the number of frames of the first image and the number of frames of the second image are reduced simultaneously.

8. An optical flow calculation device, characterized in that: The optical flow calculation device includes: An image synthesis module, configured to obtain a first image and a second image; an image processing module, configured to obtain a plurality of first corner points of the first image, obtain a plurality of second corner points of the second image, each second corner point corresponding to each first corner point, select a pixel matrix centered around the first corner point, and determine whether the second corner point corresponding to the first corner point is a valid corner point based on the number of corner points in the pixel matrix of the first corner point; A calculation module is configured to obtain an optical flow value of the second corner point according to the second corner point and an optical flow starting point of the second corner point if the second corner point matching the first corner point is a valid corner point.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; Memory; and One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the steps in the optical flow calculation method according to any one of claims 1 to 7.

10. A computer-readable medium, characterized in that A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the optical flow calculation method according to any one of claims 1 to 7.

Images