Method for identification of optical flow parameters

Abstract

FIELD: autonomous navigation.SUBSTANCE: invention relates in particular to methods of autonomous optical navigation. The method for identifying the parameters of the optical flow consists in the fact that the surface is scanned by a laser measuring system, at the output of which, as a result of recording coherent optical flows reflected from the surface, the values of their intensities and phases are formed at each point of the scanned scene, which are fed to an on-board computing device, in which the partial derivatives of the intensities and phases of the optical flows with respect to time and spatial coordinates at each point of the scanned scene are determined in real time, according to the values of which the optical flow parameter is calculated, determining the displacement of the point along the x-axis and the optical flow parameter, determining the displacement of the point along the y-axis.EFFECT: possibility of solving the problem of autonomous navigation in previously unknown terrain and reducing computational costs.1 cl

Description

[0001] The invention relates to the field of autonomous navigation, namely to autonomous optical navigation devices.

[0002] One of the effective ways to solve the problem of autonomous navigation is to determine the components of the linear and angular velocities of an object based on the parameters of the optical flow captured by a video camera during its movement.

[0003] There are known methods for identifying optical flow parameters based on taking into account the phase correlation of individual image pixels [Alba A., Arce-Santana E., Rivera M. Optical flow estimation with prior models obtained from phase correlation / / Lecture Notes in Computer Science. 2010], using various assumptions about the spatiotemporal dynamics of image brightness [Teed Z., Deng J. RAFT: Recurrent All-Pairs Field Transforms for Optical Flow / / Computer Vision - ECCV 2020, 16th European Conference, Glasgow, UK. 2020. vol. 2. pp. 402 - 419; Lin Z., Liang T., Xiao T., Wang Y., Tang Z., Yang M. FlowNAS: Neural Architecture Search for Optical Flow Estimation / / Computer Vision and Pattern Recognition. 2022. pp. 1-26], using gradient constraints from neighboring pixels [Lele S., Raychowdhury A. Fusing frame and event vision for high-speed optical flow for edge application / / 2022 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE, 2022, pp.804-808], on the assumption of equal displacement of all points of the scanned scene with subsequent approximation of the signals in the vicinity of each pixel by a quadratic form [Farnebäck G. Two-Frame Motion Estimation Based on Polynomial Expansion / / Bigun J., Gustavsson T. (eds) Image Analysis. SCIA 2003. Lecture Notes in Computer Science, vol. 2749, pp. 363-370, Springer, Berlin, Heidelberg (2003)], on the additional assumption of brightness constancy along the trajectories of object motion [Lucas BD, Kanade T. An iterative image registration technique with an application to stereo vision / / Proceedings of Imaging Understanding Workshop, 1981, pp. 121-130], on the assumption of smoothness of the optical flow over the entire image (i.e. the absence of sharp changes in the velocity field) [Horn BKP, Schunck BG Determining optical flow / / Artificial Intelligence, 1981, No. 17, pp. 185-203], etc.

[0004] The main disadvantages of these methods are, firstly, the large amount of computational costs for their implementation, which makes it impossible to identify the parameters of the optical flow in real time, and secondly, high sensitivity to the inevitable noise of measuring the intensity of the optical flow.

[0005] A device is also known that provides a solution to the problem of autonomous navigation by optical means and uses optical-electronic systems for obtaining images in the infrared range [Patent RU 2767477, 2022].

[0006] The disadvantage of this device is low noise immunity and the impossibility of solving the problem of navigation in previously unknown terrain.

[0007] The closest method to the proposed method is a method for processing an optical flow using reference indicators and matching video frames, implemented by an autonomous optical system for local positioning and navigation, containing a matrix photodetector (video camera), a primary flow processing module, a module for searching for information indicators, a video frame matching module, a module for primary filtering of contrast contours, an interface unit of external navigation sensors, an address filtering module, a module for matching contour frames, an optical odometry module, an obstacle localization module, a module for correlating instantaneous decisions, a module for integrating instantaneous and odometric decisions [Patent RU 2816087, 2024].

[0008] The disadvantage of this method is the large amount of computational costs and the impossibility of solving the problem of autonomous navigation in previously unknown terrain.

[0009] The technical result is the possibility of solving the problem of autonomous navigation in previously unknown terrain using optical measuring instruments and reducing computational costs when processing an information video stream.

[0010] The technical result is achieved in that the surface is scanned by a laser measuring system, at the output of which, as a result of recording coherent optical flows reflected from the surface, the values ​​of their intensities and phases are formed at each point of the scanned scene, which are fed to an on-board computing device, in which the partial derivatives of the intensities and phases of the optical flows with respect to time and spatial coordinates at each point of the scanned scene are determined in real time, according to the values ​​of which the optical flow parameter, which determines the displacement of the point along the x-axis,is defined as the ratio of the difference between the product of the derivative of the intensity of the optical flow with respect to time and the derivative of the phase with respect to the y coordinate and the product of the derivative of the intensity of the optical flow with respect to the y coordinate and the derivative of the phase with respect to time to the difference between the product of the derivative of the intensity of the optical flow with respect to the y coordinate and the derivative of the phase with respect to the x coordinate and the product of the derivative of the intensity of the optical flow with respect to the x coordinate and the derivative of the phase with respect to the y coordinate, and the parameter of the optical flow that determines the displacement of a point along the y axis,is defined as the ratio of the difference between the product of the derivative of the intensity of the optical flow with respect to the x-coordinate and the derivative of the phase with respect to time and the product of the derivative of the intensity of the optical flow with respect to time and the derivative of the phase with respect to the x-coordinate to the difference between the product of the derivative of the intensity of the optical flow with respect to the y-coordinate and the derivative of the phase with respect to the x-coordinate and the product of the derivative of the intensity of the optical flow with respect to the x-coordinate and the derivative of the phase with respect to the y-coordinate.

[0011] The method is based on the following conclusions.

[0012] It is known that the initial equation for identifying the optical flow parameters u,v, which make it possible to determine the linear and angular velocities of an object's motion, is the so-called basic equation of optical flow [Horn BKP, Schunck BG Determining optical flow / / Artificial Intelligence, 1981, No. 17, pp. 185-203; Lucas BD, Kanade T. An iterative image registration technique with an application to stereo vision / / Proceedings of Imaging Understanding Workshop, 1981, pp. 121-130], the basic assumption in the derivation of which is the assumption of the constancy of the intensity of the optical flow when changing adjacent video frames:

[0013] (1)

[0014] where u(x,y,t), v(x,y,t) are the current components of the brightness pattern velocity at the point (x,y) of the scanned scene;

[0015] - partial derivatives of the brightness function I(x,y,t) of the observed surface (optical flow intensity).

[0016] Since equation (1) contains two unknown variables u and v, it cannot be unambiguously resolved relative to them at a specific point in the scene. To enable the current determination of the optical flow parameters u and v at a point (x,y) in the scanned scene, we consider measurements of the reflected optical flow using a laser (coherent-optical) measuring system (LIS), for example, in the form of coherent lidars – both with a traditional scanning head and with parallel coherent optical flows [https: / / nplus1.ru / news / 2020 / 05 / 15 / coherent-lidar].

[0017] Generalizing the assumption of the constancy of the intensity of the optical flow when changing adjacent video frames in the case of its coherence to the constancy of its entire complex amplitude E(x,y,t)

[0018] where A(x,y,t) is the amplitude of the coherent optical flow, - its phase,

[0019] The equation of the complex amplitude of the optical flow E(x,y,t) is represented in a form similar to (1):

[0020]

[0021] or

[0022]

[0023] Equation (2) can be represented as a system of two equations:

[0024]

[0025] or

[0026]

[0027] The solution to system (3) is the desired vector of optical flow parameters u,v:

[0028]

[0029] Because at the point (x,y) of the scanned scene the intensity I(x,y,t)=A is recorded 2 (x,y,t) of the coherent optical flow reflected from the surface (in the LIS photodetector) and its phase (in the phase detector LIS), then in the recorded physical quantities the expression (4) of the optical flow parameters u,v after the appropriate transformations takes the form:

[0030]

[0031] The method for identifying optical flow parameters is as follows.

[0032] The LIS generates coherent optical beams that scan the surface. By recording the optical beams reflected from the surface, the LIS outputs intensity values ​​I(x,y,t) and phase. at each point (x, y) of the scanned scene, arriving at the on-board computer. In the on-board computer, the partial derivatives of the intensity and phase of the optical flows with respect to time and spatial coordinates are determined in real time at each point of the scanned scene. Also in the on-board computer, based on these partial derivatives, the optical flow parameter u is determined, which determines the displacement of a point along the x-axis - as the ratio of the difference between the product of the derivative of the intensity of the optical flow with respect to time and the derivative of the phase with respect to the y-coordinate and the product of the derivative of the intensity of the optical flow with respect to the y-coordinate and the derivative of the phase with respect to time to the difference between the product of the derivative of the intensity of the optical flow with respect to the y-coordinate and the derivative of the phase with respect to the x-coordinate and the product of the derivative of the intensity of the optical flow with respect to the x-coordinate and the derivative of the phase with respect to the y-coordinate, and the optical flow parameter v,determining the displacement of a point along the y-axis - as the ratio of the difference between the product of the derivative of the intensity of the optical flow with respect to the x-coordinate with the derivative of the phase with respect to time and the product of the derivative of the intensity of the optical flow with respect to time with the derivative of the phase with respect to the x-coordinate to the difference between the product of the derivative of the intensity of the optical flow with respect to the y-coordinate with the derivative of the phase with respect to the x-coordinate and the product of the derivative of the intensity of the optical flow with respect to the x-coordinate with the derivative of the phase with respect to the y-coordinate.

[0033] The simplicity of the computational implementation of the method ensures its effective application in autonomous optical navigation systems of objects moving in previously unknown terrain.

Images