Method for detecting surfaces, including surfaces that are not easily visible, for example transparent surfaces, semi-opaque surfaces or perforated surfaces such as grilles, meshes and nets
The method uses optical flow analysis to detect and differentiate barely visible surfaces, enhancing safety for autonomous vehicles by accurately identifying and measuring transparent or perforated surfaces, thereby reducing collision risks and operator intervention.
Patent Information
- Application Number
- PCT/EP2024/064883
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-31
- Filing Date
- 2024-05-30
- Publication Date
- 2024-12-05
AI Technical Summary
Current methods for detecting barely visible surfaces, such as transparent or perforated surfaces, are inadequate for autonomous vehicles, as they fail to accurately identify and differentiate these surfaces from textured surfaces, leading to potential collisions and requiring manual operator supervision.
A method involving the acquisition of images using an optical sensor, extraction of spherical optical flow vectors, calculation of magnitude data, and adjustment using mathematical models to detect the presence of barely visible surfaces by comparing amplitude differences between grouped data, allowing for the identification of surfaces and estimation of distances.
This method enables efficient detection and differentiation of barely visible surfaces, reducing the risk of collisions and eliminating the need for manual operator supervision, with high reliability and accuracy in identifying transparent or perforated surfaces like nets or windows.
Smart Images

Figure EP2024064883_05122024_PF_FP_ABST
Abstract
Description
DESCRIPTION Title of the invention: Method for detecting a surface including surfaces that are not very visible, for example, transparent, semi-opaque or openwork surfaces such as grids, wire mesh and nets.
[0001] The invention relates to a device and a method for detecting a barely visible surface, such as an openwork surface, in particular of the grid, wire mesh or net type, or a transparent surface, such as a glass surface. The invention also relates to the detection of such a barely visible surface, in particular when this surface is arranged between a device and a textured surface such as a wall or the ground. The invention finds application in all fields, in particular that of autonomous, marine, underwater, land and / or air vehicles having a camera and / or an optical sensor.
[0002] In the context of avoidance procedures or in a potential collision situation, it is essential for a device, such as an air, land, sea and / or underwater vehicle, to be able to detect surfaces or obstacles and to recognize its positioning in relation to these surfaces or obstacles as well as the relative distances between this device and these surfaces or obstacles, in particular in the context of autonomous navigation.
[0003] Indeed, in the case of a vehicle, when moving indoors or outdoors, certain less visible surfaces can represent a danger to the integrity of the vehicle. As an indicative example, when an aerial vehicle, such as a drone, flies indoors or outdoors, surfaces such as safety nets, grilles, wire mesh or windows represent a threat that is difficult to identify for most conventional visual detection methods.
[0004] Conventional computer vision and mapping tools fail to detect safety nets, for example. As a result, the moving device cannot take into account the less visible surfaces in its path. An operator must therefore anticipate the presence of such obstacles in the environment to avoid collisions or monitor the device's movement in real time so as to detect any openwork or transparent surfaces in time. Therefore, it is costly in terms of resources and even difficult for the operator to supervise the movement of each device, even those capable of moving autonomously.
[0005] Indeed, visual navigation methods are based on the presence of textured surfaces and a transparent or openwork surface does not have the same characteristics as a textured surface. Some studies, such as “Lidar-based glass detection for improved occupancy grid mapping” by H. Tibebu, J. Roche, V. De Silva, and A. Kondoz (Sensors, vol. 21, no. 7, p. 2263, 2021), “Mapping with reflection detection and utilization of reflection in 3D LIDAR scans” by X. Zhao, Z. Yang, and S. Schwertfeger (in IEEE International Symposium on Safety, Security, and Rescue Robotics, 2020, pp.27–33), “Cartographer glass: 2D graph SLAM framework using LIDAR for glass environments” by L. Weerakoon, GS Herr, J. Blunt, M. Yu, and N. Chopra (arXiv:2212.08633, 2022) or “Window detection from mobile LIDAR data” by R. Wang, J. Bach, and FP Ferry (Workshop on Applications of Computer Vision, IEEE, 2011, pp.58–65), propose navigation through a Lidar sensor while other documents focus on shape recognition, such as “Window detection in facades” by H. Ali, C. Seifert, N. Jindal, L. Paletta, and G. Paar (International Conference on Image Analysis and Processing, IEEE, 2007, pp. 837–842), on the combined use of several sensors of different nature, as explained in the document “Multi-sensor fusion glass detection for robot navigation and mapping” by H. Wei, X.-e. Li, Y. Shi, B. You, and Y. Xu (WRC Symposium on Advanced Robotics and Automation, IEEE, 2018, pp. 184–188), on image segmentation or direct analysis as presented in the papers “Windows detection using k-means in cielab color space” by M. Recky and F. Leberl (20th International Conference on Pattern Recognition, IEEE, 2010, pp. 356–359) or “Don't hit me! glass detection in real-world scenes” by H. Mei, X. Yang, Y. Wang, Y. Liu, S. He, Q. Zhang, X.Wei, and RW Lau (Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 3687–3696). .
[0006] It is known from the document “Deep learning of monocular depth, optical flow and ego-motion with geometric guidance for uav navigation in dynamic environments” by F. Mumuni, A. Mumuni, and CK Amuzuvi (Machine Learning with Applications, vol. 10, p. 100416, 2022) to measure optical flow and extract visual information using machine learning methods to facilitate navigation. However, these methods are limited by the data used to train the neural networks and require substantial computing power.
[0007] Document US 2009 / 225300 proposes a solution for acquiring translation optical flow vectors. However, this solution only concerns the “acquisition” part of the optical flow. No processing of the acquired optical flow is indicated, for example. Furthermore, this solution is only functional with the type of sensor disclosed in this document. Finally, document US 2009 / 225300 is limited to defining the presence of an obstacle without being able to characterize it, which can be detrimental when identification of the obstacle is necessary.
[0008] The paper "Distance estimation using a panoramic sensor based on a novel spherical optical flow algorithm" (Lan Jinhui et al) discloses a way to infer distance from an optical flow vector field. This paper also does not propose obstacle identification.
[0009] Thus, none of the documents cited above offers a viable solution for detecting surfaces and determining the presence or absence of one or more surfaces, particularly for transparent and / or openwork surfaces.
[0010] Furthermore, there is no known solution for determining the presence or absence of one or more surfaces that can be adapted to a moving device, and particularly to an autonomous vehicle.
[0011] The invention aims to overcome all or part of the problems mentioned above by proposing a method for detecting surfaces and determining the presence or absence of one or more surfaces, in particular surfaces that are not very visible such as openwork surfaces comprising grids, grilles, or nets as well as transparent surfaces such as windows.
[0012] The detection method according to the invention also makes it possible to differentiate such a surface placed between another textured surface and the moving device, the method being based on the detection of optical flow. The invention also relates to a device for detecting any surface, visible or barely visible.
[0013] To this end, the invention relates to a method for detecting the presence of at least one barely visible surface moving relative to an optical sensor. according to any translational movement, the detection method comprising the following steps: - Acquisition of images by means of the optical sensor, - Extraction of a set of spherical optical flow vectors from the acquired images, and further comprising for each optical flow vector of the set of optical flow vectors the following steps: - Calculation of a magnitude data item of the spherical optical flow vector, said data item being a physical data item of the spherical optical flow vector, - Modeling of a numerical magnitude data item of the optical flow vector by means of a first mathematical model, - Adjustment of the modeled numerical magnitude data item with respect to the magnitude data item of the spherical optical flow vector, the first model being a function of the amplitude of the optical flow, - Determination of a difference between the calculated magnitude data item of the spherical optical flow vector and the estimated numerical magnitude data item,the detection method then comprising the following steps: - Grouping the adjusted digital quantity data into a first grouping and a second grouping, the difference between each adjusted digital quantity data and the respective calculated spherical optical flow vector quantity data being positive for the first grouping, and negative for the second grouping, - Modeling the digital quantity data of the first grouping and the second grouping according to a second mathematical model, the second mathematical model being a function of an optical flow amplitude, - Comparing the optical flow amplitude between the first grouping and the second grouping, - Detecting the at least one surface as a function of the result of the comparison.,
[0014] According to one aspect of the invention, the method for detecting the presence of at least one barely visible surface moving relative to an optical sensor in any translational movement comprises the following steps: - acquisition by means of the optical sensor of images resulting from the translational movement, - obtaining a digital representation of a spherical optical flow in the form of a set of spherical optical flow vectors, from the acquired images, the obtaining step comprising for each spherical optical flow vector the following steps: - calculation of a magnitude data item of the spherical optical flow vector, said data item being a physical data item of the spherical optical flow vector, - modeling by means of a first mathematical model, of a numerical magnitude data item of the spherical optical flow vector, the first mathematical model being a function taking into account the amplitude ^^ ^^of the optical flow, - adjusting the modeled digital quantity data relative to the physical quantity data of the spherical optical flow vector, - determining an estimation error between the calculated physical quantity data of the spherical optical flow vector and the adjusted digital quantity data, the detection method comprising, after the obtaining step, the following steps: - grouping the adjusted digital quantity data into a first grouping and a second grouping, according to the estimation error between each adjusted digital quantity data and the respective calculated physical quantity data of the spherical optical flow vector, - modeling the digital quantity data of the first grouping and the digital quantity data of the second grouping according to a second mathematical model, the second mathematical model being a function taking into account a first amplitude value ^^ 1 ^^ of the flow spherical optics for the first grouping and a function taking into account a second amplitude value ^^ 2 ^^ of the spherical optical flow for the second grouping, - analysis, from the data modeled according to the second mathematical model for each grouping, of a difference between the amplitude values of the spherical optical flow of the first grouping and of the second grouping, - detection by determining the presence or absence of at least one surface facing the optical sensor, depending on the result of the analysis of the difference between said amplitude values.
[0015] According to one aspect of the invention, the detection method comprises an iteration step, for each grouping among the first grouping and the second grouping, - of adjusting the digital quantity data modeled according to the second mathematical model with respect to the quantity data of the spherical optical flow vector, for each spherical optical flow vector of the set of spherical optical flow vectors, - of determining an estimation error between the digital quantity data according to the second mathematical model and the calculated quantity data, for each spherical optical flow vector of the set of spherical optical flow vectors, - of grouping according to two distinct groups of quantity data according to the estimation error, - of modeling the digital quantity data of each of the two groups according to a predefined mathematical model, - of analyzing,from the data modeled according to the predefined mathematical model for each grouping of each of the two groups, of a difference between the amplitude values of the spherical optical flow for each grouping of each of the two groups, - and determining the presence of a surface that is not very visible compared to the optical sensor, based on the result of the analysis of the difference between said amplitude values.
[0016] According to one aspect of the invention, the detection method comprises a step of filtering the digital quantity data following the adjustment step.
[0017] According to one aspect of the invention, the step of obtaining comprises a step of obtaining a set of planar optical flow vectors and a step of backprojecting the set of planar optical flow vectors following the step of obtaining the set of planar optical flow vectors.
[0018] According to one aspect of the invention, the first mathematical model is defined according to the following function: where ^^ ^^ represents the amplitude of the spherical optical flow, ^^ represents a correction factor for the motion of the optical sensor relative to the at least one surface and ^^ ^^ represents a magnitude of the spherical optical flow at an instant ^^ of displacement and at an angle ^^ relative to the optical sensor.
[0019] According to one aspect of the invention, the step of adjusting the digital quantity data comprises an approximation step according to the iterative least squares method.
[0020] According to one aspect of the invention, the adjustment step comprises an additional step of canceling the adjustment if the iterative least squares method does not converge after a predefined number of steps or if the estimation error determined between the calculated spherical optical flow vector magnitude data and the adjusted numerical magnitude data is greater than ten percent of the value of the calculated spherical optical flow vector magnitude data.
[0021] According to one aspect of the invention, the detection method comprises, following the adjustment step, a step of selecting a magnitude data zone, the data zone being defined according to the following equation: 0.7� ^^ ^^ ^^ ^^ ^^ 3( ^^ ∗ ^^ ) � < | ^^ ^^( ^^) | < 1.3� ^^ ^^ ^^ ^^ ^^ 3( ^^ ∗ ^^ ) �
[0022] According to one aspect of the invention, the first mathematical model is defined according to the following function: where ^^ ^^ represents the amplitude of the spherical optical flow, ^^ ^^ represents a magnitude of the spherical optical flow at an instant ^^ and at an angle ^^ relative to the optical sensor, m represents a coefficient depending on the angle of incidence between the optical sensor and the at least one surface and the orientation deviation between the at least one surface and the normal of the at least one surface and σ represents a correction factor for the movement of the optical sensor relative to the at least one surface.
[0023] According to one aspect of the invention, the step of adjusting the digital quantity data comprises an approximation step according to the direct least squares method.
[0024] According to one aspect of the invention, the step of determining the presence or absence of at least one surface facing the optical sensor comprises an additional step of determining at least two surfaces if the difference resulting from the analysis between the amplitude values of the spherical optical flow of the first grouping and the second grouping is greater than a determination threshold. The determination threshold is ideally ten percent.
[0025] According to one aspect of the invention, the detection method comprises a step of estimating a distance between the optical sensor and the at least one barely visible surface, the distance between the optical sensor and the at least one surface being defined according to the following function: where ^^ represents the velocity of the optical sensor relative to the at least one observed surface and ^^^^ ^^ ^^ ^^ ^^represents an amplitude value of the spherical optical flow of the at least one observed surface.
[0026] The invention also relates to an optical flow sensor configured to detect a spherical optical flow, the optical flow sensor comprising a device for detecting a low-visibility surface, the detection device being configured to implement the steps of the detection method having at least one of the characteristics previously described.
[0027] The invention will be better understood and other advantages will appear on reading the detailed description of an embodiment given by way of example, a description illustrated by the attached drawing in which:
[0028] [Fig.1] Figure 1 represents a schematic view of a method for detecting a surface;
[0029] [Fig.2] Figure 2 represents a schematic view of a variant of the method for detecting a surface;
[0030] [Fig.3A] Figure 3A shows a representative graph of a detection of a single surface by means of the detection method;
[0031] [Fig.3B] Figure 3B represents a representative graph of a detection of two superimposed surfaces by means of the detection method;
[0032] [Fig.4] Figure 4 represents a schematic view of a device for detecting a barely visible surface arranged between the detection device and another surface.
[0033] For the sake of clarity, the same elements will have the same references in the different figures.
[0034] Figure 1 represents a schematic view of a method 1 for detecting at least one surface, and in particular at least one barely visible surface, from an optical sensor. In other words, the method is a method for determining the presence or absence of at least one surface in front of the optical sensor.
[0035] To do this, the detection or determination method 1 is based on the measurement of an optical flow and therefore requires that the optical sensor, included in the device, moves relative to this at least one surface. Furthermore, the detection method 1 is also capable of detecting such a barely visible surface positioned between the optical sensor and any other surface, for example a textured surface, so that a superposition of the surfaces is observable from the point of view of the optical sensor.
[0036] For the purposes of the invention, a low-visibility surface may be defined as a) a surface having a certain transparency. According to a preferred configuration, the low-visibility surface is a transparent surface and has a certain visual disparity in its transparency with the presence of visual contrasts, for example the presence of impurities, inclusions, dust or defects on the surface. This surface may also consist of b) an openwork surface such as a net, a grid, a wire mesh.
[0037] For the purposes of the invention, a textured surface may be a surface which, unlike the less visible surface, has a certain texture. In other words, the textured surface has a certain opacity compared to the less visible surface. As an illustrative example, the textured surface is an opaque surface or a surface having significant visual disparities.
[0038] Optical flow is defined as the apparent motion of objects, surfaces, and contours in a visual scene, caused by the relative motion between an observer, the device, and the observed scene.
[0039] The method according to the invention makes it possible to differentiate several barely visible surfaces when they are superimposed without being in contact.
[0040] The method of the invention also makes it possible to identify these less visible surfaces being positioned in front of a textured surface. It is not necessary for the textured surface to be flat.
[0041] The detection method 1 comprises a first step 10 of acquiring images by means of an optical sensor. During the acquisition step 10, the optical sensor performs image captures or photographs according to a predefined frequency. More precisely, the image captures or photographs can be taken according to an acquisition period of between 10 milliseconds and 100 milliseconds.
[0042] Images are acquired by the optical sensor and are produced by any translational movement of the optical sensor.
[0043] Following the acquisition of images in step 10 by the optical sensor, a second step allows extracting a set of spherical optical flow vectors from the acquired images or photographs. More precisely, it is possible to define points on an acquired image and to follow the displacement of these points on the images successively acquired by the optical sensor. Therefore, an optical flow vector can be defined as the displacement of a targeted point between two successively acquired images. During step 11, the detection method 1 makes it possible to obtain from the acquired images a set of spherical optical flow vectors, that is to say a set of spherical optical flow vectors for example in an area of interest of the image. As an indicative example, three aligned spherical optical flow vectors can be obtained per acquired image. This set of spherical optical flow vectors can take the form of a line or a diffuse field.
[0044] In other words, step 11 allows obtaining a digital representation of a spherical optical flow in the form of the set of spherical optical flow vectors.
[0045] The obtained optical flow, as well as the set of optical flow vectors, is a spherical optical flow resulting from the translational movement of the optical sensor, i.e. an optical flow defined in three dimensions which is obtained by the translational movement of the optical sensor relative to the at least one surface.
[0046] The translation of the optical flow sensor relative to the at least one surface is any translation. In other words, the movement must be a rectilinear movement relative to the at least one surface and the optical flow sensor is therefore not limited in its translational movement.
[0047] Preferably, the set of spherical optical flow vectors can be directly extracted from the acquired images. To do this, different methods are known. According to a preferred configuration, it can be envisaged to use, during step 11, the LUCAS KANADE algorithm, as described in the document “An iterative image registration technique with an application to stereo vision” by BD Lucas and T. Kanade (IJCAI'81: 7th international joint conference on Artificial intelligence, vol.2, 1981, pp.674–679), in conjunction with a corner detector according to the Shi-Thomasi method, presented in the document “Good features to track” by J. Jianbo and C. Tomasi (IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1994, pp. 593–600) to extract the Optical Flow. The method selects certain points on an image that are defined as corners, and attempts to follow them into the next image thus forming spherical optical flow vectors. This process is highly configurable: it is possible to define the number of searched points, their quality as well as the number of filter levels, which refine the final measurement of the resulting optical flow, but more searched points require more computing power.
[0048] From then on, a resulting optical flow is obtained, this resulting optical flow is defined according to a set of spherical optical flow vectors between two successive images.
[0049] Furthermore, it can also be considered to extract a spherical optical flow vector field to improve the measurement of the resulting optical flow.
[0050] Alternatively, it may be envisaged to obtain a planar optical flow to which processing or filtering is carried out as shown in FIG. 2. The obtaining step 11 may comprise a step of extracting or obtaining a set of planar optical flow vectors 112 by means of a conventional flat retina camera for example, and a step of backprojection 114 of the set of planar optical flow vectors following the step of obtaining the set of planar optical flow vectors 112.
[0051] Indeed, the optical sensor can be defined to operate only in two dimensions. From then on, there is extraction of a set of vectors or a field of planar vectors.
[0052] And, at this back-projection step 114, the measurement of the optical flow is planar, since it is taken from a two-dimensional image. However, the most useful representation of the resulting optical flow is the spherical representation. The only difference then is the fact that the resulting planar optical flow is projected onto a sphere, in order to obtain a resulting spherical optical flow measurable in radians per second. To do this, a matrix expression of the back-projection, taking into account the intrinsic parameters of the optical sensor such as the pixel size, the focal length or the resolution can be applied.This matrix expression of the backprojection is applied to all the planar optical flow vectors detected during step 112 to thereby transform a two-dimensional planar representation of the planar optical flow vectors and thus of the extracted resulting optical flow into a three-dimensional spherical representation of the spherical optical flow vectors and thus of the extracted resulting optical flow. Any expression. classical backprojection matrix expression can be considered. Nevertheless, details on the computation of the backprojection matrix expression can be given in the paper “A general approach for egomotion estimation with omnidirectional images” by R. Vassallo, J. Santos-Victor, and H. Schneebeli (Proceedings of the IEEE Workshop on Omnidirectional Vision 2002. Held in conjunction with ECCV'02, 2002, pp. 97–103 as well as in the book “Robotics, Vision and Control” by P. Corke (Springer, 2017).
[0053] The backprojection step thus allows the calculation of a backprojection matrix at each observed point or corner, thus making it possible to switch from a planar view to a spherical view of the optical flow and of each of the extracted optical flow vectors.
[0054] The backprojection step can be replaced by any processing or filtering step that allows switching from a representation of planar optical flow vectors to a representation of spherical optical flow vectors.
[0055] Then follows, for each spherical optical flow vector of the set of vectors, a calculation step 12 of a magnitude datum of the spherical optical flow vector. The magnitude datum of the spherical optical flow vector is a physical datum of the optical flow vector. Indeed, the optical flow vector can be defined according to its direction and its norm. However, the magnitude datum is the norm of the spherical optical flow vector or only an axial component of the spherical optical flow vector or a combination of several components of the spherical optical flow vector. The magnitude datum of the vector is therefore a physical datum of the optical flow vector. Physical datum is understood to mean data extracted directly from the optical flow vector by physical measurement during step 11.Furthermore, the definition of the magnitude data, according to the norm of the vector or according to at least one component of the vector depends solely on a desire to simplify as much as possible the future numerical modeling of this data. Indeed, choosing only one component makes it possible to define only a translational movement and therefore to eliminate any other movement by this hypothesis, simplifying the numerical calculation. Conversely, defining the magnitude data as the norm of the vector makes it possible to take into consideration any movement of the optical sensor with respect to the at least one observed surface but increases the complexity of the numerical calculations. And, defining the magnitude data as the norm of the. vector allows to take into account any movement of the optical sensor relative to at least one observed surface but increases the complexity of the numerical calculations.
[0056] In other words, the magnitude of the spherical optical flow vector may be the norm of the spherical optical flow vector. Alternatively, the magnitude of the spherical optical flow vector may be a component of the spherical optical flow vector along one of the three axes of the frame formed by the optical flow sensor. The magnitude of the optical flow vector may also be a combination of two components of the spherical optical flow vector along two of the three axes of the frame formed by the optical flow sensor.
[0057] Preferably, the magnitude data of the spherical optical flow vector is the norm of the vector.
[0058] In other words, for each corner or point analyzed, an optical flow vector can be obtained and a magnitude data of the vector can be estimated. And all of these spherical optical flow vectors are grouped to form a spherical optical flow.
[0059] A set of spherical optical flow vectors can then be obtained and, for each vector, the magnitude data can be obtained. It is then possible to numerically transcribe these optical flow vectors, and to use a numerical expression of the optical flow coinciding with the spherical optical flow vectors.
[0060] The detection method 1 comprises, for each optical flow vector of the set, a modeling step 13 using a first mathematical model. This mathematical model is a function of an amplitude ^^ ^^ of the optical flow. The digital optical flow vector magnitude data is thus mathematically modeled so as to be equivalent to each spherical optical flow vector magnitude data obtained during the calculation step 12.
[0061] Modeling step 13 makes it possible to obtain a digital representation of the spherical optical flow and in particular of the magnitude data of the spherical optical flow for each observed point or corner.
[0062] More precisely, the resulting spherical optical flow can be represented numerically using its measured spherical optical flow magnitude data. ^^ ^^ ( ^^ ) , with knowledge of the corner considered as well as the movement of the optical sensor between the two image acquisitions.
[0063] Each magnitude data of the optical flow vector can then be calculated using different methods.
[0064] According to one embodiment, when the optical sensor moves parallel to the at least one surface.
[0065] In other words, according to the first configuration, the optical sensor moves in translation, parallel to the plane which is analyzed, namely the at least one surface. The modeling step 13 then makes it possible to estimate the spherical optical flow measured in the direction of movement of the optical sensor.
[0066] The spherical optical flow measured in the context of any translation can be defined according to the formula introduced by Koenderink, JJ, van Doorn, AJFacts on optic flow. Biol. Cybern.56, 247–254, 1987:
[0067] By defining the observation angle ( ^^ ^^ ) relative to the optical axis of the optical sensor, the magnitude data of the measured spherical optical flux can then be defined as: where ^^ represents the distance from the orthogonal between the previously detected corner and the optical sensor. In other words, D represents the distance between the optical sensor and the at least one surface as a function of the defined theoretical flux; where V represents the measurable velocity component with respect to the motion of the resulting optical flux � ^ � ^ ^^ between two images acquired successively in the direction parallel to the at least one surface; where ^^ represents the angle between the optical sensor and the point considered, namely the observed corner; where ^^ ^^ represents the distance between the optical sensor and the at least one observed surface; where ^^ represents the pointing direction of the optical sensor.
[0068] In the presence of a translational movement parallel to the at least one surface to be detected, the value of the resulting optical flow can be approximated to the curve of the magnitude of the spherical optical flow measured by a function of the type ^^^^ ^^ ^^ ^^ 3 ( ^^ ∗ ^^) with ^^ ^^ and k being parameters of the model, ^^ ^^ represents an amplitude value of the spherical optical flow measured at the corner considered. k is a coefficient value introduced to compensate for approximations. The addition of this correction parameter makes it possible to reduce the modeling error without significantly increasing the processing time.
[0069] Given the measurement deviations such as translational movement of the optical sensor relative to the surface to be detected, the model advantageously incorporates a corrective factor k.
[0070] The correction factor k is between a value equal to 0 and a maximum value equal to 3. Preferably, the correction factor k is between 0.8 and 2.2 and ideally, the correction factor k is equal to 1.
[0071] For information purposes, according to the first embodiment and by defining the magnitude data of the optical flow vector as the component of the optical flow vector along the direction of movement of the device carrying the optical sensor, the modeling step 13 may comprise a calculation step 131, represented in FIG. 2, of the numerical magnitude data, preferably the norm of the optical flow vector. According to this first configuration of parallel movement relative to the at least one surface. The component of the optical flow vector along the direction of movement of the device carrying the optical sensor, that is to say, the magnitude data of the optical flow vector is then calculated according to the following function: where ^^ ^^represents an amplitude value of the spherical optical flow measured at the corner considered and k represents the correction factor of the movement of the optical sensor relative to the at least one surface at an instant ^^ during the movement and at an angle ^^ relative to the optical sensor.
[0072] In other words, the first mathematical model is defined according to the following function:
[0073] Alternatively, when the movement is only in translation relative to the at least one surface and has an angle of incidence between the optical sensor and the at least one surface, the magnitude of the optical flow vector can be calculated, during step 131, according to the following formula: where ^^ ^^ represents the amplitude of the measured spherical optical flow, ^^represents a value of the optical flow at an instant ^^ and at an angle ^^ relative to the optical sensor, m represents a coefficient depending on the angle of incidence between the optical sensor and the at least one surface and the orientation deviation between the at least one surface and the normal of the at least one surface and σ represents a correction factor for the movement of the optical sensor relative to the at least one surface.
[0074] Therefore, the second mathematical model is defined according to the following function:
[0075] This formula thus has the advantage of better representing, in a numerical way, the magnitude of the resulting optical flow observed previously but requires more significant computational resources. Indeed, in the presence of any angle of incidence between the optical sensor and at least one surface, that is to say a biased shooting angle, the use of a cosine type function is generally not very functional. Indeed, the angle of incidence adds complexity to the model.
[0076] This function also has the advantage of tending towards zero when the iteration increases. Thus, this function follows the theory that optical flow evolves inversely proportional to distance as stated with the formula of the Koenderink and van Doorn formula, and as described according to the formula stated above.
[0077] In other words, the optical flow calculation step 131 makes it possible to approximate the digital magnitude data of the optical flow, namely the estimated digital magnitude of the optical flow, to the value of the magnitude of the vector of the resulting spherical optical flow analyzed previously.
[0078] Furthermore, any type of numerical model that allows obtaining a “bell-shaped” function with a relatively flat top can be considered. As an indicative example, a trigonometric function based on the sine can be considered, as well as a polynomial function or a Gaussian function.
[0079] As an indicative example, the first mathematical model can be a polynomial function of type: where a,b,c,d,e represent parameters of the polynomial.
[0080] Then follows, for each optical flow vector of the set of optical flow vectors, a step 14 of adjusting the estimated digital optical flow vector magnitude data relative to the calculated spherical optical flow vector magnitude data.
[0081] In the adjustment step 14, the numerical magnitude data model is approximated to the value of each optical flow vector magnitude calculated in the calculation step 12. To do this, the adjustment step 14 uses the least squares method. Indeed, the least squares method optimizes the optical flow magnitude curve to minimize the sum of the square of the errors on each approximated point, namely each corner.
[0082] According to a preferred aspect, the adjustment step 14 comprises an approximation step 142 according to the iterative least squares method with a limit on the number of iterations. In other words, the adjustment step 14 may comprise an additional step of canceling the adjustment 144 if the iterative least squares method does not converge after a predefined number of iteration steps. As an indicative example, this predefined number of iteration steps may be between 2000 and 20000 and is preferably defined at 10000 iterations.
[0083] Alternatively, it may be envisaged that the adjustment step 14 comprises an approximation step 142 according to the direct least squares method. This method has the advantage of being more directive than the iterative least squares method but has the disadvantage of being more expensive in terms of computing power. In addition, the direct least squares method also has the disadvantage of sometimes not converging.
[0084] Alternatively, any numerical optimization method that minimizes the residuals between the measured optical flow quantities and the modeled numerical quantities can be considered. As an indicative example, the gradient descent method can also be considered.
[0085] Following the adjustment step 14, the detection method 1 comprises, for each spherical optical flow vector, a step of determining an estimation error 15 between the calculated magnitude data of the spherical optical flow vector and the adjusted numerical magnitude data. More precisely, this involves determining the estimation error between the sum of the errors between the calculated magnitude data of the spherical optical flow vector and the adjusted numerical magnitude data. The step of determining the estimation error 15 thus makes it possible to determine the estimation error or the distance between the magnitude of the measured spherical optical flow vector and the value of the numerical magnitude according to the first mathematical model at the observed point.
[0086] According to a preferred but optional embodiment of the invention, if the step of determining the estimation error 15 on all the data gives an estimation error less than a threshold, for example five percent, then it is concluded that the optical flow comes from a single surface.
[0087] Following the step of determining the deviation 15, the detection method 1 comprises a step of grouping 16 the adjusted digital quantity data into a first grouping and a second grouping. For example, using the deviation between each adjusted digital quantity data and the respective calculated spherical optical flow vector quantity data is positive for the first grouping and negative for the second grouping.
[0088] In other words, the grouping step 16 makes it possible to segment the adjusted digital quantity data into a first grouping and a second grouping, according to the estimation error between each adjusted digital quantity data and the respective calculated spherical optical flow vector physical quantity data.
[0089] The grouping step 16 thus makes it possible to separate all of the optical flow digital magnitude data into two groups or clusters, namely the first grouping and the second grouping.
[0090] In other words, the grouping step 16 involves generating the hypothesis that at least two surfaces are superimposed in front of the optical sensor and that therefore the numerical distribution is large enough to observe at least two surfaces. In other words, the values of spherical vector magnitudes are relatively far from each other.
[0091] Following this grouping 16, the detection method 1 comprises a new estimation step carried out on the digital magnitude data of the first grouping and the second grouping 17 according to a second mathematical model, the second mathematical model being a function of an amplitude ^^ ^^ of the optical flow. It is then a question, during step 18, of estimating, in a similar way to estimation step 13, the numerical quantity data of the first grouping according to the second mathematical model and the numerical quantity data of the second grouping according to the second mathematical model. From then on, the numerical quantity data of the first grouping is then calculated according to the following second mathematical model: � ^^ 1 ^^ ( ^^)� = ^^ 1 ^^ ^^ ^^ ^^ 3 ( ^^1 ∗ ^^) represents the magnitude of the optical flow of the first optical flow group, ^^ 1 ^^represents an amplitude value of the spherical optical flow measured at the corner considered for the first grouping and k1 represents a first corrective coefficient of the movement of the optical sensor relative to the at least one surface at an instant ^^ and at an angle ^^ relative to the optical sensor, and the digital quantity data of the second grouping is calculated according to the following second mathematical model: where ^^2 represents the numerical magnitude of the optical flow of the second grouping, ^^ 2 ^^ represents an amplitude value of the spherical optical flow measured at the corner considered for the second grouping and k2 represents a second corrective coefficient of the movement of the optical sensor relative to the at least one surface at an instant ^^ and at an angle ^^ relative to the optical sensor.
[0092] Preferably, the first corrective coefficient k1 is equal to the second corrective coefficient k2.
[0093] It should be noted that the second mathematical model can be equivalent to the first mathematical model, as indicated above. However, any other mathematical model can be considered as, for example, a second mathematical model based on the following function:
[0094] In other words, modeling step 17 makes it possible to segment the digital magnitude data of the first grouping and the digital magnitude data of the second grouping according to the second mathematical model. The second mathematical model is a function taking into account a first amplitude value ^^ 1 ^^ of the spherical optical flow for the first grouping and a function taking into account a second amplitude value ^^ 2 ^^ of spherical optical flow for the second grouping.
[0095] It is then necessary to establish, for each surface whose hypothesis is established, that is to say two surfaces, the coefficient m as a function of the angle of incidence between the optical sensor and the at least one surface and the deviation in orientation between the at least one surface and the normal of the at least one surface and the correction factor σ of the movement of the optical sensor relative to the at least one surface.
[0096] Thus, preferably, the second mathematical model is identical to the first mathematical model. However, it can also be envisaged that the second mathematical model is different from the first mathematical model or even another mathematical model of the same "type", that is to say bell-shaped for example. The term "bell-shaped" defines any function whose graphical representation is a curve whose summit is relatively flat.
[0097] Then follows a comparison step 18 of the amplitude ^^ ^^of the optical flow between the first grouping and the second grouping. This involves verifying the hypothesis stated previously during grouping step 16, namely whether at least two surfaces are identifiable. More precisely, it is then a question of comparing the amplitude of the optical flow according to the second numerical model for the first grouping with the amplitude of the optical flow according to the second numerical model for the second grouping.
[0098] In other words, step 18 allows an analysis, from the data modeled according to the second mathematical model for each grouping, of a difference between the amplitude values of the spherical optical flow of the first grouping and the second grouping.
[0099] If there is little or no difference, i.e. less than a threshold that can be defined, between the value of the amplitude of the optical flow of the first grouping ^^ 1 ^^and the value of the amplitude of the optical flow of the second grouping ^^ 2 ^^ then the distribution is centered, which reflects the presence of a single surface facing the optical sensor. In other words, the optical flow of the first grouping is identical to the optical flow of the second grouping and there is actually only one surface facing the optical sensor, invalidating the basic hypothesis. The threshold can, for example, be set at ten percent.
[0100] There is then a determination 19 of the presence of a barely visible surface and the detection of this.
[0101] Conversely, if a gap is observable then it is possible to state that either there is no surface facing the optical sensor, or at least two superimposed surfaces are facing the optical sensor.
[0102] In order to be able to know if at least two superimposed surfaces are facing the optical sensor, the comparison step 18 can comprise an additional step of determining at least two surfaces 182. More precisely, highlighting a difference in the value of the amplitude of the optical flow of the first grouping ^^ 1 ^^ and the value of the amplitude of the optical flow of the second grouping ^^ 2 ^^ may sometimes not be enough to identify whether it is a situation with at least two surfaces or no surfaces. This is because noise may be included in the first grouping or in the second grouping so that a small difference may be observed between the value of the optical flow amplitude of the first grouping ^^ 1 ^^ and the value of the amplitude of the optical flow of the second grouping ^^ 2 ^^ .
[0103] The determination step 182 induces an additional condition, if the comparison of the amplitude ^^ ^^of the optical flow between the first grouping and the second grouping is included in a determination interval, then two surfaces are detected.
[0104] The determination threshold is defined as a percentage value of difference between the value of the amplitude of the optical flow of the first grouping ^^ 1 ^^ and the value of the amplitude of the optical flow of the second grouping ^^ 2 ^^ . According to one embodiment of the invention, this threshold percentage is ten percent. In other words, if there is more than ten percent difference between the value of the amplitude of the optical flow of the first grouping ^^ 1 ^^ and the value of the amplitude of the optical flow of the second grouping ^^ 2 ^^ percent, then there is determination of at least two surfaces.
[0105] According to a preferred variant, the determination threshold is five percent.
[0106] Conversely, if there is less than ten percent difference between the value of the optical flow amplitude of the first grouping ^^ 1 ^^ and the value of the amplitude of the optical flow of the second grouping ^^ 2 ^^ , that is to say that the deviation is small, then it is a centered distribution with the presence of noise explaining the small phase shift between the first group of optical flow and the second group of optical flow.
[0107] Detection method 1 then only distinguishes one surface.
[0108] According to a preferred variant, the step of determining two surfaces includes an additional condition on the deviation between the adjusted values in each grouping and the measured values of each grouping. If the adjustment errors relating to each grouping are less than a second threshold, for example fifteen percent, then it is concluded that there are indeed at least two observed surfaces. Otherwise, the distribution is said to be too heterogeneous to conclude that there are surfaces.
[0109] In other words, if the dispersion of the amplitude values grouped in the first grouping or in the second grouping exceeds fifteen percent, then the distribution in the first grouping or in the second grouping is itself heterogeneous. This dispersion means the presence of at least two surfaces and that it is therefore necessary to repeat the determination method 1 for the first grouping or for the second grouping, that is to say to segment the amplitude data into more than two groupings. This is a step of verifying the distribution of the amplitudes of a grouping among, for example, the first grouping or the second grouping.
[0110] Moreover, if the value of the amplitude of the optical flow of the first grouping ^^ 1 ^^ and the value of the amplitude of the optical flow of the second grouping are far from each other, then the distribution is heterogeneous even with application of the first model and the second model.
[0111] Alternatively, this conclusion may arise from the fact that the two groupings are too dispersed.
[0112] This then results in an absence of surface or excessively noisy digital quantity data, thus preventing the detection of planes facing the optical sensor; the detection step 19 then does not distinguish any surface facing the optical sensor.
[0113] In other words, step 19 is a step for determining the number of surfaces detected by the optical sensor and by the detection method 1. This number can be equal to 0 or greater than 1.
[0114] Furthermore, if the comparison step 18 highlights the fact that more than one surface is detected, i.e. when the distribution is not sufficiently centered to validate the presence of a single surface, it is possible to envisage, by means of the detection method 1, a reiteration of the detection process on each of the two groupings, namely the first grouping and the second grouping.In other words, if more than one surface is detected in step 19, i.e. at least two superimposed surfaces, the detection method 1 can repeat, respectively for the first grouping and for the second grouping, the steps of adjusting 14 the digital quantity data of the second mathematical model with respect to the calculated quantities, determining a difference 15 between the digital quantities according to the second mathematical model and the calculated quantity data, grouping 16 according to two distinct groups of quantity data according to the difference, modeling 17 the digital quantity data according to another mathematical model, comparing 18 the digital quantity data according to this other mathematical model of the two distinct groups of quantity data and detecting 19 at least one surface for each of the two distinct groups of data. And. This iterative loop of steps 14 to 19 for each new group of digital data makes it possible to determine the number of surfaces facing the optical sensor.
[0115] In other words, the detection method 1 comprises, following the detection step 19, an iteration step, for each grouping among the first grouping and the second grouping, - of the adjustment 14 of the digital quantity data modeled according to the second mathematical model with respect to the quantity data of the spherical optical flow vector, for each optical flow vector of the set of optical flow vectors, - of determination of a difference 15 between the digital quantity data according to the second mathematical model and the calculated quantity data, for each optical flow vector of the set of optical flow vectors, - of grouping 16 according to two distinct groups of quantity data according to the sign of the difference, - of modeling 17 of the digital quantity data of each of the two groups according to a predefined mathematical model, - of comparison 18 of the amplitude ^^ ^^of the optical flow of the two groups of data of distinct quantities, and other criteria according to variants, - and detection 19 of at least one surface according to the result of the comparison.
[0116] It may be envisaged, optionally, to process, according to a preprocessing step 10', as shown in FIG. 2, the optical data acquired during the acquisition step 10, in order to extract a central band on each image of 20 to 200 pixels wide. Ideally, the central band is approximately 60 pixels wide. This preprocessing step 10' thus makes it possible to reduce the computational cost in the detection method 1.
[0117] Furthermore, the extraction of the set of optical flow vectors during the obtaining step 11 must make it possible to extract a three-dimensional optical flow. Thus, the optical flow sensor can advantageously be a camera, and preferably a “fish eye” type camera or a spherical optical flow sensor.
[0118] The detection method 1 may comprise, following the adjustment step 14, a step of removing extreme points 146 from the digital magnitude data of the optical flow vector. In other words, the step of removing the extreme points 146 allows a removal of the aberrant points in the first mathematical model. More precisely, the step of removing the extreme points 146 allows a selection of a band or zone of value around the curve defining the digital magnitude data of the optical flow vector by removing all the digital data which deviate too much from the corresponding detected resulting optical flow value.
[0119] In other words, during the step of deleting the extreme points 146, it is a matter of deleting from the first mathematical model the magnitude data judged to be too far from the expected result, that is to say the extreme points, reflecting for example erroneous local measurements or problems linked to the optical sensor.
[0120] As an indicative example, this value zone or band may be defined as a band of values between seventy percent and one hundred and thirty percent of the value of the adjusted numerical magnitude of the spherical optical flow. In other words, the step of removing the extreme points 16 may comprise a step of selecting 146, shown in FIG. 2, a magnitude zone, the magnitude zone being defined according to the following optical flow equation: 0.7� ^^ ^^ ^^ ^^ ^^3( ^^ ∗ ^^)� < | ^^ ^^( ^^)| < 1.3� ^^ 3^^ ^^ ^^ ^^ ( ^^ ∗ ^^)�.
[0121] The value area or band around the curve can be broadened and defined as a band of values between twenty-five percent and one hundred and seventy-five percent of the value of the refined spherical measured optical flow quantity.
[0122] In other words, the area of accepted quantities is then defined according to the following optical flow equation:0.25� ^^ ^^ ^^ ^^ ^^3( ^^ ∗ ^^)� < | ^^ ^^( ^^)| < 1.75� ^^ 3^^ ^^ ^^ ^^ ( ^^ ∗ ^^)�.
[0123] Alternatively, the value area or band around the curve can be reduced and defined as a band of values between ninety percent and one hundred and ten percent of the value of the measured spherical refined optical flow magnitude.
[0124] In other words, the area of accepted quantities is then defined according to the following optical flow equation:0.9� ^^ ^^ ^^ ^^ ^^3( ^^ ∗ ^^)� < | ^^ ^^( ^^)| < 1.1� ^^ 3^^ ^^ ^^ ^^ ( ^^ ∗ ^^)�.
[0125] The selection step 148 thus allows filtering of the digital quantities of the vector.
[0126] If other functions are used to model the optical flow, the definition of the band becomes, for a fitted function ^^( ^^) :0.25 ∗ ^^( ^^) < | ^^ ^^( ^^)| < 1.75 ∗ ^^( ^^) .
[0127] In addition, it may be envisaged to calculate the distance separating the optical sensor from the at least one surface, if this at least one surface is detected during step 19.
[0128] Similarly, for each detected surface, it is possible to estimate the relative angle ^^ between the observer, namely the optical sensor, and the targeted plane.
[0129] In a situation where only one surface is identified, it is possible to measure the distance separating the optical sensor from said surface facing it. To do this, the detection method 1 may comprise a step 20 of estimating a distance ^^^^ ^^ ^^ ^^ between the optical sensor and said detected surface. Indeed, from the formula stated previously, it is possible to link the distance between the optical sensor and the detected surface and the value of the magnitude of the optical flow vector calculated:
[0130] The distance ^^^^ ^^ ^^ ^^between the optical sensor and the detected surface is then defined according to the following function: where ^^ represents the speed of the optical sensor relative to the detected surface and ^^^^ ^^ ^^ ^^ ^^represents an optical flow amplitude value, which is equal to the optical flow amplitude value of the first grouping ^^ 1 ^^ or even to the value of the amplitude of the optical flow of the second grouping ^^ 2 ^^ since the distribution is centered.
[0131] In a situation where at least two surfaces are detected, the value of the optical flow amplitude that has the smallest value thus defines the greatest distance separating the detected surface from the at least two detected surfaces of the optical sensor. And, the value of the optical flow amplitude that has the largest value thus defines the smallest distance separating the detected surface from the at least two detected surfaces of the optical sensor.
[0132] Advantageously, if two surfaces have been detected during step 19, such as for example an openwork surface located in front of a textured surface, the detection method 1 can allow the estimation of a distance ^^ ^^ ^^ ^^ between the optical sensor and the perforated surface and a distance ^^^^ ^^ ^^ ^^between the optical sensor and the textured surface.
[0133] The distance ^^ ^^ ^^ ^^ between the optical sensor and the openwork surface is then defined according to the following function: where ^^ represents the speed of the optical sensor relative to the detected openwork surface and ^^^^ ^^ ^^ ^^represents the largest optical flow amplitude value between the optical flow amplitude value of the first grouping ^^ 1 ^^ or to the value of the amplitude of the optical flow of the second grouping ^^ 2 ^^ .
[0134] And, the distance ^^^^ ^^ ^^ ^^between the optical sensor and the textured surface is defined according to the following function: where ^^ represents the speed of the optical sensor relative to the detected openwork surface and ^^^^ ^^ ^^ ^^ ^^represents the smallest optical flow amplitude value between the optical flow amplitude value of the first grouping ^^ 1 ^^or to the value of the amplitude of the optical flow of the second grouping ^^ 2 ^^ .
[0135] Alternatively, in the context of any translation and a function different from those presented, the formula becomes:^^ ^^ ^^ ^^ ^^ = ^^ ^^ / | ^^ ^^0 | with K, a parameter depending on the type of translation, and | ^^ ^^0 | the value of the optical flow norm along the main observation direction.
[0136] The relative orientation angle between the optical sensor and the surface ^^ is defined according to several equations. For example, the function: maybe used.
[0137] This equation describing the evolution of the component along the X axis of the spherical optical flow can be used (for example by a least squares method) to approximate the angle ^^. Within the framework of this function, A and ^^ ^^are parameters dependent on the movement of the optical sensor. Et� ^^ ^^_ ^^ ( ^^)� represents the component of the spherical optical flow along the X axis.
[0138] The following study of other components of the spherical optical flow vector is possible and leads to other equations. For example, the study following the evolution of the component along the Y axis, leads to the following equation:
[0139] Other equations exist and can be used alternatively, such as those published in F. Expert and F. Ruffier 2015 Bioinspir. Biomim. 10026003, but they are based on fewer measurements and are therefore less reliable.
[0140] Obtaining the parameter ^^ is therefore possible by solving the equation describing the evolution of the component along the X axis of the spherical optical flow or, in a preferred variant, via the use of an optimization method, such as the least squares method, to obtain ^^.
[0141] The detection method 1 was applied as previously indicated according to the first configuration. Two case studies were set up, a first situation with an optical sensor placed in front of a textured surface. And, a second situation setting up the same optical sensor and the same textured surface with, between the optical sensor and the textured surface, an openwork surface such as a net.
[0142] For both situations, the relative angle between the optical sensor and the at least one surface is zero, and the distance separating them is constant.
[0143] Figure 3A thus represents the numerical result of the detection of a single surface by means of detection method 1 while Figure 3B represents the numerical result of the detection of two surfaces, namely the textured surface and the openwork surface, superimposed by means of detection method 1.
[0144] Thus, it is observable in Figure 3A a distribution of the digital quantities of optical flows, which are defined according to this indicative example as the absolute values of the component following the direction of movement of the optical sensor, more centered, demonstrating the presence of a single surface allowing the projection of optical flow in the direction of the optical sensor, namely the textured surface alone, whereas in Figure 3B, the distribution of the digital quantities of optical flow is much more scattered and shows the presence, by the use of the detection method 1, of two surfaces generating optical flows, namely the textured surface and the superimposed openwork surface.
[0145] It should be noted that each surface detection using detection method 1 results from measurements taken with a single pair of two successive images. However, during a passage or trajectory several detections or acquisitions can be accumulated so as to obtain several times an estimate of digital quantities of optical flows and refine the detection by increasing the number of samples.
[0146] Thus, other experiments were also implemented according to the same parameters as stated previously. For each of these experiments, the number of images per trajectory and the number of usable images per trajectory was also indicated: Flight 1 2 3 4 5 6 7 8 9 Case AAAAAABBB Image 30 30 30 30 30 30 20 20 20 Image 25 30 26 28 25 28 18 20 16 usable Number of 0 3 2 3 1 1 17 20 16 detection Percentage 100 90 93 89 96 96 94 100 100 correct estimate Determination AAAAAABBB of Case Vol 10 11 12 13 14 15 Case BBBBBB Image 30 30 30 30 30 30 Image 28 27 29 28 26 26 usable Number of 28 27 29 27 26 26 detection Percentage 100 100 100 97 100 100 correct estimate Determination BBBBBB of Case
[0147] Case A represents a situation with an optical sensor placed in front of a textured surface while case B represents a situation with the same optical sensor and the same textured surface with, between the optical sensor and the textured surface, a net.
[0148] Thus, these results highlight the fact that detection method 1 presents a reliability of the order of 90% by taking only two images successive and still allows detection of a net if several pairs of images are integrated during a pass. Detection method 1 detects the net with a rate of absence of detection, less than 5% and also has an erroneous detection rate, less than 10%. Again with only two images used. The use of several detections allows to further increase the precision of detection method 1 and of the device going to 100%
[0149] The invention also relates to an optical flow sensor 100, represented in FIG. 4, configured to capture images in the direction of a textured surface 110. The optical flow sensor 100 comprises a detection device 120 of an openwork surface 130 between the optical flow sensor and the textured surface 110. The detection device 120 is thus configured to implement the steps of the detection method 1.
[0150] In other words, the detection device 120 comprises: - A module for acquiring images by the optical flow sensor 100. The acquisition module makes it possible to implement the acquisition step 10. The acquisition module can also implement the preprocessing step 10' if necessary. - A module for obtaining a set of spherical optical flow vectors from the acquired images. - A module for calculating a magnitude data item for the spherical optical flow vector 12. - A module for adjusting the estimated numerical magnitude data item with respect to the magnitude data item for the spherical optical flow vector. - A module for determining a difference between the calculated magnitude data item for the spherical optical flow vector and the estimated numerical magnitude data item.- A module for grouping the adjusted digital quantity data into a first grouping and a second grouping, the difference between each adjusted digital quantity data and the respective calculated spherical optical flow vector quantity data being positive for the first grouping, the difference between each digital quantity data. adjusted and the respective calculated spherical optical flow vector magnitude data being negative for the second grouping. - A module for estimating the numerical magnitude data of the first grouping and the second grouping according to a second mathematical model, the second mathematical model being a function of an amplitude ^^ ^^ optical flow. - And an amplitude comparison module ^^ ^^ of the optical flow between the first grouping and the second grouping and detection of the at least one surface depending on the result of the comparison.
[0151] The detection device may also include a processor or an electronic card for analyzing the magnitude data of each optical flow vector to enable real-time in-flight detections.
[0152] Finally, the detection device 120 also comprises a module for estimating the distance between the optical sensor and the at least one surface detected according to step 20.
[0153] The detection of a non-opaque or semi-transparent surface such as a net or a window, by an optical sensor mounted on a device such as a drone for example, is thus made possible thanks to the detection method 1. The nature of the optical sensor can be arbitrary. As an indicative example, the optical sensor can be a low-cost global shutter camera. The detection method 1 thus has the advantage of allowing more efficient and safer movement for the device, without an operator needing to first check for the presence of such obstacles in the area. The detection method 1 can also be added to an avoidance algorithm. In addition, with a confidence of more than 90% in direct measurements and nearly 100% in the estimation of the global trajectory, any semi-transparent surface is detected in a few steps with a limited number of images.
[0154] The detection method 1 is thus similar to a method for determining the presence or absence of a surface observed by the optical sensor by means of a measurement of spherical optical flow vectors resulting from the translational movement of the optical sensor as well as the number of surfaces superimposed relative to the point of view of the optical sensor by developing a number of models digital and comparing them with the visual data obtained by the optical sensor. The observed surface is understood as the external part of a body or an object. The surface can be an openwork, semi-transparent or even opaque surface. Preferably, the surface is a window, a grid, a wire mesh or even a net, as stated previously.
Claims
CLAIMS 1. Method for detecting (1) the presence of at least one surface (110, 130), facing an optical sensor (120) moving in any translational movement relative to the at least one surface (110, 130), the detection method (1) comprising the following steps: - acquiring (10) by means of the optical sensor (120) images resulting from the translational movement, - obtaining (11) a digital representation of a spherical optical flow in the form of a set of spherical optical flow vectors, from the acquired images, the obtaining step comprising for each spherical optical flow vector the following steps: - calculating (12) a magnitude data item of the spherical optical flow vector, said data item being a physical data item of the spherical optical flow vector, - modeling (13) by means of a first mathematical model, a digital magnitude data item of the spherical optical flow vector,the first mathematical model being a function taking into account the amplitude ^^, ^^of the optical flow, - adjusting (14) the modeled digital quantity data relative to the physical quantity data of the spherical optical flow vector, - determining (15) an estimation error between the physical quantity data of the calculated spherical optical flow vector and the adjusted digital quantity data, the detection method (1) comprising, after the obtaining step, the following steps: - grouping (16) the adjusted digital quantity data into a first grouping and a second grouping, according to the estimation error between each adjusted digital quantity data and the respective calculated physical quantity data of the spherical optical flow vector, - modeling (17) the digital quantity data of the first grouping and the digital quantity data of the second grouping according to a second mathematical model, the second mathematical model being a function taking into account a first amplitude value ^^。1 ^^ of the spherical optical flow for the first grouping and a function taking into account a second amplitude value ^^ 2 ^^ of the spherical optical flow for the second grouping, - analysis (18), from the data modeled according to the second mathematical model for each grouping, of a difference between the amplitude values of the spherical optical flow of the first grouping and the second grouping, - detection by determining (19) the presence or absence of at least one surface facing the optical sensor, depending on the result of the analysis of the difference between said amplitude values.
2. Detection method (1) according to claim 1, comprising an iteration step, for each grouping among the first grouping and the second grouping if the dispersion of the amplitudes in a grouping among the first grouping or the second grouping is greater than fifteen percent, - adjustment (14) of the digital quantity data modeled according to the second mathematical model with respect to the quantity data of the spherical optical flow vector,for each spherical optical flow vector of the set of spherical optical flow vectors, - determining (15) an estimation error between the digital magnitude data according to the second mathematical model and the calculated magnitude data, for each spherical optical flow vector of the set of spherical optical flow vectors, - grouping (16) into two distinct magnitude data groups according to the estimation error, - modeling (17) the digital magnitude data of each of the two groups according to a predefined mathematical model, - analyzing (18), from the data modeled according to the predefined mathematical model for each grouping of each of the two groups, a difference between the amplitude values of the spherical optical flow for each grouping of each of the two groups, - and detecting by determining (19) the presence of at least one surface facing the optical sensor,depending on the result of the analysis of the difference between said amplitude values.
3. Detection method (1) according to claim 1 or 2, comprising a step of filtering (142) the digital quantity data following the adjustment step (14).
4. Detection method (1) according to one of claims 1 to 3, in which the obtaining step (11) comprises a step of obtaining a set of planar optical flow vectors (112) and a backprojection step (114), of the set of planar optical flow vectors following the step of obtaining the set of planar optical flow vectors (112).
5. Detection method (1) according to one of claims 1 to 4, in which the first mathematical model is defined according to the following function: where ^^ ^^ represents the amplitude of the spherical optical flow, ^^ represents a correction factor for the motion of the optical sensor relative to the at least one surface and ^^^^represents a magnitude of the spherical optical flow at a time ^^ and at an angle ^^ relative to the optical sensor.
6. Detection method (1) according to one of claims 1 to 5, wherein the step of adjusting (14) the digital magnitude data item comprises an approximation step (142) according to the iterative least squares method.
7. Detection method (1) according to claim 6, wherein the adjustment step (14) comprises an additional step of canceling (144) the adjustment if the iterative least squares method does not converge after a predefined number of steps or if the estimation error determined between the calculated magnitude data item of the spherical optical flow vector and the adjusted digital magnitude data item is greater than ten percent of the value of the calculated magnitude data item of the spherical optical flow vector. 8.Detection method (1) according to claim 3, comprising, following the adjustment step (14), a step of selecting (146) a magnitude data area, the data area being defined according to the following equation:0.7� ^^ ^^ ^^ ^^ ^^3( ^^ ∗ ^^)� < | ^^ ^^( ^^)| < 1.3� ^^ 3^^ ^^ ^^ ^^ ( ^^ ∗ ^^)�9. Detection method (1) according to one of claims 1 to 4, in which the first mathematical model is defined according to the following function:. where ^^ ^^represents the amplitude of the spherical optical flow, ^^ represents a magnitude of the spherical optical flow at a time ^^ and at an angle ^^ relative to the optical sensor, m represents a coefficient depending on the angle of incidence between the optical sensor and the at least one surface and the deviation in orientation between the at least one surface and the normal of the at least one surface and σ represents a correction factor for the movement of the optical sensor relative to the at least one surface.
10. Detection method (1) according to one of claims 1 to 5, wherein the step of adjusting (14) the digital magnitude data comprises an approximation step according to the direct least squares method.
11. Detection method (1) according to claim 10, wherein the step of determining (19) the presence or absence of at least one surface facing the optical sensor comprises an additional step of determining at least two surfaces (182) if the difference resulting from the analysis (18) between the amplitude values of the spherical optical flow of the first grouping and the second grouping is greater than ten percent.
12. Detection method (1) according to one of the preceding claims, comprising a step of estimating (20) a distance ^^^^ ^^ ^^ ^^ between the optical sensor (120) and the at least one surface (110, 130), the distance ^^^^ ^^ ^^ ^^ between the optical sensor (120) and the at least one surface (110, 130) being defined according to the following function: where ^^ represents the speed of the optical sensor relative to the at least one observed surface (110, 130) and ^^^^ ^^ ^^ ^^ ^^ represents an amplitude value of the spherical optical flow of the at least one observed surface (110, 130).
13. Detection method (1) according to one of the preceding claims, comprising a step of estimating (20) a distance ^^^^ ^^ ^^ ^^ between the optical sensor (120) and the at least one surface (110, 130), the distance ^^^^ ^^ ^^ ^^ between the optical sensor (120) and the at least one surface (110, 130) being defined according to the following function: ^^ ^^ ^^ ^^ ^^ = ^^ ^^ / | ^^ ^^0 | with K, a parameter depending on the type of translation and | ^^ ^^0| the value of the optical flow standard along the main observation direction of the optical sensor.
14. Detection method (1) according to one of the preceding claims, comprising a step of determining the relative orientation angle ^^ between the optical sensor (120) and the at least one determined surface (110, 130), the orientation angle ^^ being defined according to the following formula: where ^^ and ^^ ^^ represent parameters of the movement of the optical sensor (120),� ^^ ^^_ ^^ ( ^^)� represents the component of the spherical optical flow along an axis.
15. Optical flow sensor (100) configured to detect a spherical optical flow, the optical flow sensor (100) comprising a detection device (120) of at least one surface (110, 130), the detection device (120) being configured to implement the steps of the detection method (1) according to any one of the preceding claims.
16. Optical flow sensor (100) according to the preceding claim, wherein the optical flow sensor is a camera, preferably a fisheye type camera.
Citation Information
Patent Citations
Translational Optical Flow Sensor
US20090225300A1
