Method for automatically realigning the navigation of an optronic system moving around a navigation zone

Abstract

The present invention relates to a method for automatically realigning the navigation of an optronic system (18) moving around a navigation zone, the method comprising, over time, the following steps: estimating, by a navigation device, a current position and a current orientation for the optronic system (18); acquiring an imaging datum; processing the imaging datum, the processing step comprising segmenting the imaging datum so as to highlight characteristic elements in the imaging datum; determining a set of realigning corrections of the navigation device; and updating and maintaining the navigation of the navigation device based on the determined set of corrections.

Description

[0001] TITLE: Automatic recalibration method for the navigation of an optronic system operating in a navigation zone

[0002] The present invention relates to a method for automatically recalibrating the navigation of an optronic system operating within a navigation zone. The present invention also relates to such an optronic system and an associated vehicle.

[0003] The field of the invention relates to the autonomous recalibration of vehicle navigation, in particular military vehicles, also addressed under the more general term PNT for position navigation timing in English (translated into French as position navigation temps).

[0004] In particular, the aim is to solve the problem of navigation information drift in a vehicle equipped with an inertial measurement unit (IMU) and an optronic sensor in a contested environment. A contested environment is defined as one potentially without access to GNSS (Global Navigation Satellite System) signals, without means of communication, without an electromagnetic emission sensor (vs. discretion and power consumption), without collaborating elements in the environment (beacons, specific markings, etc.) as in US 2007 / 0276590 A, and without the use of pre-existing ground-based imagery as in US 2012 / 0300020 A. Such imagery is not available or updated in hostile areas (e.g., enemy territory).

[0005] It is desired that the navigation recalibration can be carried out both in structured and unstructured environments, in the broadest sense, on all types of terrain on which the vehicle carrying the optronic system is likely to move on the surface of the planet.

[0006] The state of the art provides solutions for navigation assistance in passenger vehicles and civilian transport, with applications and research dedicated to autonomous driving. These vehicles are generally equipped with appropriate sensors, systematically using a GNSS receiver and frequently employing active sensors such as LiDAR, radar, and / or active imaging sensors. The operating environment is often structured (urban). Obstacle avoidance and route planning functions are often emphasized, and off-road driving constraints are of little interest in the civilian sector. These solutions are therefore not suitable for contested environments, particularly military ones where discretion is paramount.

[0007] There is therefore a need for an automatic navigation recalibration solution for an optronic system that can be used in a contested environment, without GNSS signals, without emission and without reception of electromagnetic waves emitted from objects or a third-party network, and without emitting electromagnetic waves (in the radar and optronic domains from ultraviolet to infrared).

[0008] To this end, the present description relates to a method for the automatic recalibration of the navigation of an optronic system operating within a navigation zone. The navigation zone comprises characteristic elements, the characteristic elements comprising channels. The optronic system comprises the following elements integrated within the optronic system: a navigation device capable of estimating over time at least one position and orientation for the optronic system and of guiding the movement of the optronic system according to the estimated position and orientation; an imaging device capable of acquiring images of the environment as seen from the optronic system; a memory in which a graph of the characteristic elements of the navigation zone as seen from the sky is stored; and a computing unit. The method is implemented by the optronic system and comprises, over time, the following steps: estimation,by the navigation device, of a current position and current orientation for the optronic system, the acquisition of imaging data by the imaging device, the imaging data being an image or a stream of images of the environment seen from the optronic system, the processing of the imaging data by the computing unit, the processing step including the segmentation of the imaging data so as to highlight characteristic elements, called identified elements, on the imaging data, the determination, by the computing unit, of a set of recalibration corrections for the navigation device, the determination step including:,

[0009] • the matching of at least one identified element on the image data with a characteristic element of the graph as a function of the current position and current orientation of the optronic system, • the determination of a set of corrections of the current position and current orientation of the optronic system, to realign the at least one identified element on the image data with the corresponding characteristic element of the graph, and the updating and maintenance of the navigation of the navigation device as a function of the set of corrections determined.

[0010] Depending on other advantageous aspects, the process includes one or more of the following characteristics, taken individually or in all technically possible combinations:

[0011] - during the step of determining a set of corrections, at least one transverse realignment correction of the current position and orientation of the optronic system relative to the identified element under consideration is determined;

[0012] - the characteristic elements of the graph include crossings of paths and / or paths with specific curvatures, during the step of determining a set of corrections, when at least one identified element is a crossing of paths and / or a path with a specific curvature, the matching is also carried out according to the shape of the identified element, and the set of corrections includes at least one longitudinal recalibration correction of the position and current orientation of the optronic system with respect to the identified element considered;

[0013] - during the step of determining a set of corrections, the matching includes the search for characteristic elements of the graph, called candidate elements, within a predetermined perimeter around the current position, the element associated with the identified element being chosen from among the candidate elements according to a selection criterion;

[0014] - the selection criterion stipulates that: the associated element is the candidate element closest to the current position, or in the presence of several candidate elements at a distance less than a predetermined distance from the current position, the associated element is the candidate element generating a lower association cost when implementing a probabilistic approach, preferably the probabilistic approach implementing a calculation of Mahalanobis distances between each candidate element and the current position;

[0015] - the graph in memory was previously obtained by processing one or more ortho-images of the navigation area so as to highlight the characteristic elements of the navigation area on the ortho-image(s) and to extract the shape of said characteristic elements to form the graph;

[0016] - the characteristic elements are highlighted on the ortho-image(s) by a deep learning algorithm, for example an algorithm implemented by a neural network;

[0017] - a digital terrain model of the navigation area is also stored in the memory of the optronic system, during the step of determining a set of corrections, the matching also being a function of the digital terrain model;

[0018] - in the absence of identified elements on the image data, the steps of determining a set of recalibration corrections, and of updating and maintaining the navigation are interrupted, said steps being resumed as soon as at least one element is identified on the image data;

[0019] - at the end of the navigation update and maintenance step, at least the corrected position of the optronic system is stored in the optronic system memory to be reused later in the event of a possible interruption of the steps of determining a set of recalibration corrections, and of updating and maintaining the navigation.

[0020] This description also relates to an optronic system evolving in a navigation zone, the navigation zone comprising characteristic elements, the characteristic elements comprising channels, the optronic system being configured to implement a process as described above, the optronic system comprising the following elements: a navigation device capable of estimating over time at least one position and orientation for the optronic system and of guiding the movement of the optronic system according to the estimated position and orientation, an imaging device capable of acquiring images of the environment seen from the optronic system, a memory in which is stored a graph of the characteristic elements of the navigation zone in view of the sky, and a computing unit.

[0021] This description also relates to a vehicle, such as a land vehicle, comprising an optronic system as described above. The invention will become clearer upon reading the following description, given solely by way of non-limiting example, and made with reference to the drawings in which:

[0022] [Fig. 1] Figure 1 is a schematic representation of an example of a vehicle equipped with an optronic system operating in a navigation zone comprising characteristic elements,

[0023] [Fig 2] Figure 2 is a schematic representation of an example of an optronic system comprising elements integrated into said system,

[0024] [Fig 3] Figure 3 is an example of a portion of a navigation zone graph, [Fig 4] Figure 4 is a flowchart of an example of the implementation of an automatic navigation recalibration process for an optronic system operating within a navigation zone, and

[0025] [Fig 5] Figure 5 is a schematic representation of an example illustrating a processing carried out on an image data in order to highlight characteristic elements (routes) on the image data.

[0026] In the following description, the absolute (geographic) orientation of an object in a scene is defined as the vector joining the coordinate system to the object; the associated direction being the line containing this vector. This absolute orientation is characterized by angles expressed relative to a geographic reference. The most commonly used are the azimuth angle, which expresses the orientation in a locally horizontal plane (tangent to the ellipsoid associated with the geoid) relative to the local geographic meridian, and the elevation angle (or inclination angle), which expresses the orientation in a vertical plane relative to the locally horizontal plane. A compass is typically used to measure azimuth. An inclinometer is typically used to measure elevation. A relative orientation is defined with respect to another reference orientation (i.e., an angular difference between two orientations), characterized by the bearing angle in the horizontal plane and the elevation angle in the vertical plane.A goniometer typically allows the measurement of a deposit and a site.

[0027] The attitude of an element is defined as the information that allows the element to be fully oriented in a frame of reference covering the 3 dimensions of geographic space (for example, in a minimal way with the 3 Euler angles of roll, pitch and yaw).

[0028] A scene 10 is illustrated as an example in Figure 1. A scene designates a theater of operations, that is, the location where an action takes place. The scene is therefore an extensive space with sufficient dimensions to allow the action to unfold. The scene is typically an outdoor space. Scene 10 belongs to a navigation zone. The navigation zone is, for example, a region of space extending over several kilometers. The navigation zone is also referred to as the area of ​​operations, mission area, or scene.

[0029] Scene 10 is, for example, a road network. Alternatively, scene 10 is a portion of space outside the road network.

[0030] The navigation area includes characteristic elements with known geographic coordinates (latitude, longitude) and known precision.

[0031] Characteristic elements are ground-level features. More specifically, ground-level features include roads, particularly traffic lanes. Characteristic elements thus form a road network. Examples of characteristic elements include roads, highways, paths, and trails.

[0032] The feature features have known geographic coordinates (latitude, longitude). Optionally, the elevation of each feature feature is also known. Preferably, the accuracy of the latitude and longitude coordinates of each feature feature is known from the quality metadata attached to an orthophoto representing the navigation area as seen from above. Similarly, the elevation accuracy is known from the metadata of a digital terrain model indicating the elevations at ground points in the navigation area used in the vicinity of the feature feature.

[0033] In the example illustrated by Figure 1, the characteristic elements are a first straight track V1, a second track V2 with a non-zero curvature, and a junction C formed by the intersection of the first track V1 and the second track V2.

[0034] As illustrated by the example in Figure 1, an optronic system 18 evolves in scene 10.

[0035] The optronic system 18 is preferably mounted on a vehicle 19.

[0036] Vehicle 19 is, for example, a land vehicle as shown in Figure 1. Vehicle 19 is, for example, a military type, such as an assault tank or an infantry fighting vehicle (IFV). Such a military vehicle is specifically designed to carry multiple weapons and to protect the operator(s) inside the vehicle.

[0037] Alternatively, vehicle 19 is an aerial vehicle (aircraft, drone) or a maritime vehicle (boat).

[0038] As illustrated in Figure 2, the optronic system 18 comprises at least the following elements: a navigation device 20, an imaging device 21, a memory 22, and a processing unit 28. Optionally, the optronic system 18 also includes a display element 30. Preferably, these elements are integrated into the optronic system 18. The term "integrated" means that the elements are physically and / or software-based within the optronic system 18. Such elements thus form a single unit within the optronic system 18.

[0039] The navigation device 20 is capable of estimating over time at least one position and orientation for the optronic system 18 and of guiding the movement of the optronic system 18 according to the estimated position and orientation.

[0040] The navigation device 20 is, for example, an inertial component, for example an inertial measurement unit.

[0041] The imaging device 21 (also referred to as "sensor" in the description) is designed to acquire images of the environment seen from the optronic system 18, i.e. images of the scene 10.

[0042] Typically, the native images acquired by the imaging device 21 can be resampled to a specific geometry or projection. The images are composed of pixels identified by pixel coordinates. Each pixel corresponds to a direction characterized by its angular values ​​(bearing, elevation) in a frame of reference linked to the optronic sensor (imaging device 21). The sensor mounting on the vehicle is determined, for example, by a calibration procedure, which allows the angular information from a direction in the sensor's frame of reference to be transferred to a vehicle frame of reference. In the case of multiple distributed sensors, the sensor mounting allows the directions recorded by each sensor to be processed in a common vehicle frame of reference.

[0043] Preferably, the images acquired by the imaging device 21 are panoramic or near-panoramic images. An image is considered panoramic when it images a scene over 360° in azimuth. The elevation range is, for example, between 75° and -15°. A near-panoramic image is a panoramic image with occultation zones. An occultation zone is an angular sector in azimuth for which the scene is not imaged. An occultation zone extends, for example, at most 25° in azimuth and over the entire elevation range of the image.

[0044] In practice, elevation ranges can be limited by objects in the image. Image processing can also reduce the areas actually acquired when these areas correspond to regions with low a priori information, such as elevated sites representing the sky, for example.

[0045] Advantageously, the imaging device 21 is capable of operating in several spectral bands, for example, in the visible and infrared.

[0046] The imaging device 21 is a passive device. This allows for electromagnetic discretion and limits energy consumption. The imaging device 21 is, for example, composed of a single sensor (camera). This allows for optimal spatial and temporal coherence for the acquired images.

[0047] Alternatively, the imaging device 21 is formed by a set of sensors (cameras) and their images are then time-stamped and geometrically adjusted in order to extract useful information as if one had an instantaneous view of the scene (where applicable panoramic).

[0048] Preferably, the imaging device 21 is an omnidirectional imaging device, that is to say an imaging device capable of providing 360° images in azimuth and up to 180° in elevation.

[0049] In another variant, the imaging system 21 does not image the entire 360° due to masking or an architecture composed of a set of non-overlapping distributed cameras. As mentioned previously, the images are then described as quasi-panoramic.

[0050] In another example, the imaging device 21 includes one or more MINERVA sensors developed by THALES.

[0051] Memory 22 contains in memory a graph G of the characteristic elements of the navigation area in view of the sky.

[0052] The G graph can be extracted prior to the mission by a third-party system, or even by on-board processing in the sensor using satellite images or drones capable of acquiring and georeferencing images from the sky.

[0053] Alternatively, the graph G can be constructed during the mission (in real time or delayed) from other elements, for example an orthoimage.

[0054] Graph G was not obtained from a 3D model acquired from the ground and requiring prior acquisition by land means (vehicle or pedestrian).

[0055] In an example of implementation, graph G represents a skeletonization of the characteristic elements of the navigation area, that is to say that only the contours (or the shape) of the characteristic elements are represented on graph G. Graph G thus represents a road network (made up of roads, highways, paths (rural and forest paths and tracks in desert environments), or trails).

[0056] An example of a graph G is illustrated by figure 3.

[0057] Preferably, as illustrated in Figure 3, the images used to construct graph G are preprocessed to add geometric information (lane width), intersection radii, sections with large radii of curvature, sections with steep slopes, etc. The coordinates of the vertices of graph G allow for the pre-calculation of length and planar and vertical orientation information. The topology of vertices associated with more than two edges is typical of L-shaped curve junctions, T-shaped junctions with three branches, X-shaped junctions with four branches, and more. For example, for a junction with a crossing, the relative angles between the different branches are calculated and constitute a strong signature of the vertex, especially when a long path is required to find a vertex with the same crossing topology.

[0058] Thus, in an example, the graph G includes, in particular, one or more of the following pieces of information: geometric properties, including: o The lengths and widths of the roads, as well as their planar orientations, which can be deduced from the positions of the vertices in the graph G. o The slope of the path between two nodes, based on the distance and altitude differences of the vertices. o The distance and direction of the next intersections can optionally be used to determine the spatial and / or temporal proximity for a future longitudinal realignment. semantic properties, including: o The class nature of the edge between vertices (roads, path). o The confidence, expressed as a real number on [0, 1], in belonging to the indicated class. topological properties, including: o The nature of a junction between edges of the graph G, described by connectivity analysis, outlines its geometric shape class (such as L, O, T, X, Y, see Fig. 3).o The topological signature corresponding to the different angular differences between branches of the graph G and in particular at the level of a crossroads.

[0059] Memory 22 also includes, associated with graph G, the coordinates of the characteristic elements represented on graph G (latitude, longitude, preferably altitude). The accuracy errors associated with these data are preferably also provided; they are obtained in particular from the quality of the source data used to construct the graph.

[0060] In one implementation example, the graph G in memory 22 was previously obtained by processing one or more orthoimages of the navigation area to highlight the characteristic features of the navigation area on the orthoimage(s) and extract the shape of these characteristic features to form the graph G. Preferably, the characteristic features are highlighted on the orthoimage(s) by a deep learning algorithm, for example, an algorithm implemented by a neural network. In one implementation example, the deep learning algorithm provides grayscale images indicating the confidence level of finding routes with certainty or, conversely, the potential for the lowest confidence level. The resulting grayscale image can be represented in logarithmic form to best highlight the less clear access routes on the orthoimage.The image obtained imaging the pathways then lends itself to various classic image processing techniques, for example, morphological, connectivity, skeletonization or learning, in order to construct a graph G.

[0061] In one example implementation, the graph G is developed before the first movement of the optronic system 18, for example using an image seen from the sky already available at the planetary scale, or a part of an image that would have been acquired more recently by an aerial vector equipped with optronic sensors and a means of producing geo-referenced ortho-images on the area of ​​evolution of the optronic system 18. This ability to construct the graph G, including in short loops, gives a dynamic aspect to the application.

[0062] In one implementation example, graph G was consolidated using, for example, multiple images, including multi-date / multi-season images. The merging of several graphs still results in a graph that can be used in the same way as a graph obtained from a single instance of a single modality.

[0063] Preferably, memory 22 includes ephemeris files giving the position of the sun or a catalogue of astrometric data giving the position of other bodies in the solar system according to the dating.

[0064] Preferably, memory 22 also includes a geometric image capture model. This image capture model allows a pixel in one of the images from the optronic system 18 to be associated with a spatial direction described by two angles (bearing, elevation) in the coordinate system of the frame attached to the optronic system 18. This frame of the optronic system 18 has an attitude relative to the local geographic coordinate system, and the three (Euler) angles characterizing this attitude constitute three of the six unknowns of the complete system pose to be determined. Thus, any point of a reference element in the scene with coordinates in an optronic image has a relative direction in the frame of the optronic system 18.

[0065] The computing unit 28 is suitable for receiving data from other elements of the optronic system 18, including images from the imaging device 21, data stored in memory 22 and data from the navigation device 20.

[0066] Calculation unit 28 is, for example, a processor.

[0067] In one example, the computing unit 28 interacts with a computer program product that includes an information storage medium. The information storage medium is readable by the computing unit 28.

[0068] A readable information medium is a suitable medium for storing electronic instructions and capable of being connected to a computer system bus. Examples of readable information media include optical discs, CD-ROMs, magneto-optical discs, ROMs, RAMs, EPROMs, EEPROMs, magnetic cards, optical cards, and USB flash drives. The computer program product, comprising program instructions, is stored on the information medium.

[0069] The computer program can be loaded onto the computing unit 28 and leads to the implementation of steps of an automatic recalibration process of the navigation of an optronic system 18 evolving in a navigation area, when the computer program is implemented on the computing unit 28 as will be described later in the description.

[0070] The display element 30 is suitable for displaying images from the imaging device 21 and / or data stored in memory 22.

[0071] The display element 30 is, for example, a display, such as an OLED screen.

[0072] The operation of the optronic system 18, which involves the implementation of an automatic navigation recalibration process for an optronic system 18 evolving in a navigation zone, will now be described with reference to the flowchart in Figure 4, and to Figure 5, which illustrates an example of a step in the process.

[0073] The recalibration process is implemented by the optronic system 18.

[0074] The steps of the process are implemented over time (real time). Preferably, the different steps of the process are implemented continuously, for example at the acquisition frequency of the imaging device 20.

[0075] At initialization, the position and orientation of the optronic system 18 are provided to the navigation device 20. For example, the initial position and orientation are given either by a GNSS (if it is reliable at the location of the start of the mission), or by bearings from georeferenced landmarks, or by any other means of static positioning such as bearing.

[0076] ESTIMATION OF A CURRENT POSING The process includes a step 90 of estimation, by the navigation device 20, of a current position and a current orientation for the optronic system 18. These estimations are subject to errors depending on the drift of the navigation device 20. It is this drift that we seek to correct in the following steps of the process.

[0077] IMAGING DATA ACQUISITION

[0078] The process includes a step 100 of acquiring imaging data by the imaging device 21.

[0079] Image data is an image or image stream of the environment as seen from the optronic system 18.

[0080] Preferably, the image data is a panoramic (or quasi-panoramic) image, or a stream of panoramic (or quasi-panoramic) images of scene 10.

[0081] Image acquisition can take place with the aircraft stationary or in motion. The speed of movement must be compatible with the integration time of the imaging device to avoid blurring the image. For example, a speed of approximately 50 km / h can be achieved with an integration time of a few milliseconds. Preferably, the position maintained by the inertial measurement unit (IMU) should be acquired simultaneously with the image acquisition.

[0082] In one example, the imagery data is constructed from videos of optronic sensors distributed around vehicle 19. The images from said sensors are time-stamped and assembled using beam-fitting processing to establish a panoramic image or a stream of panoramic images of the scene 10.

[0083] Knowledge of the image assembly, describing their position and attitude within a common vehicle reference frame, can be refined as needed to meet performance objectives. This is achieved by favoring the use of observations comprised of: matching image primitives in overlapping image areas; to improve accuracy, small point primitives with sub-pixel precision are used.

[0084] Matching objects with known ground coordinates to their corresponding image coordinates;

[0085] A mixture of the two previous types of observations: relative and absolute.

[0086] IMAGING DATA PROCESSING

[0087] The process includes a step 110 of processing the imaging data by the computing unit 28. The processing step 110 includes segmenting the imaging data so as to highlight and extract characteristic elements, called identified elements, from the imaging data.

[0088] In one implementation example, segmentation is performed by a deep learning algorithm. Preferably, segmentation is performed by a semantic segmentation algorithm.

[0089] The segmentation can be, for example, segmentation by class, including a "lane" class, where "lanes" could be roads, highways, paths, or trails. The outlines of other vehicles detected during this segmentation can be used as potential road masking information, and the semantic information can be applied to several successive images.

[0090] Other element classes can also be used, for example, building classes, ground feature classes, tree / forest classes, and pole / pylon classes. These classes are "static" and therefore likely to be present in an orthophoto taken from the sky. Still other classes can be considered, for example, a sky class not present in an orthophoto taken from the sky, and "mobile" vehicle and human classes probably not present in a previously taken orthophoto taken from the sky.

[0091] The extraction is, for example, performed by an algorithm that extracts the contours of characteristic elements. For example, the contours of characteristic elements correspond to the intersections between each highlighted characteristic element and the highlighted elements for different element classes (road, field, vegetation, buildings, etc.).

[0092] An example of segmentation is shown in the top image of Figure 5 and of extraction in the bottom image of Figure 5.

[0093] In particular, each identified characteristic element has a direction which is obtained as a function of the pixel coordinates of the characteristic element on the panoramic data since each pixel corresponds to a direction characterized by its angular values ​​(bearing, site).

[0094] DETERMINATION OF A SET OF REALIZATION CORRECTIONS

[0095] The process includes a step 120 of determining, by the computing unit 28, a set of corrections for recalibrating the navigation device 20.

[0096] The determination step involves matching at least one identified element in the image data with a characteristic element of graph G based on the current position and orientation of the optronic system. In one implementation example, the matching process involves searching for characteristic elements of graph G, known as candidate elements, within a predetermined perimeter around the current position. The element associated with the identified element is then selected from among the candidate elements based on a selection criterion.

[0097] In one implementation example, the selection criterion stipulates that the associated element is the candidate element closest to the current position.

[0098] In one implementation example, the distance between the current solution and nearby branches of graph G can be ambiguous. Therefore, as a complement or alternative, when several candidate elements are located less than a predetermined distance from the current position, the selection criterion stipulates that the associated element is the candidate element resulting in the lowest association cost when implementing a probabilistic approach.

[0099] Preferably, the probabilistic approach implements a calculation of Mahalanobis distances between each candidate element and the current position.

[0100] More specifically, the following sub-steps are implemented, for example:

[0101] A windowing substep, based on the quality of the current solution, allows us to search for vertices of the graph G that are compatible in distance, which allows us to identify vertices and edges that are candidates for vehicle traffic.

[0102] We deduce the distances of the candidate edges to the current position solution, and also consider the orientations of these edges, as well as the errors associated with these quantities.

[0103] We use a probabilistic data association (PDA). The decision to associate the current position with one of the candidate branches of the graph G is made using a Mahalanobis distance that takes into account the distances of the branches of the graph G to the solution. Each candidate edge 'G' of the graph for finding the vehicle is assigned a cost c G sum of normalized distances and orientation.

[0104] C G = (PG ~ PN ^P PG ~ PN) + 0G - ^ / A' OG - V N )

[0105] Expression in which:

[0106] P G represents one of the candidate graph positions, obtained by perpendicular projection onto the edge of the current navigation position P N

[0107] - HAS P corresponds to the position covariance sum of the position error covariance of navigation indicated by filter A P)V and the covariance of error of the vehicle's position as it moves along this edge A Cv This last one inherits, in particular, the metadata of the geographic products used: planar position error of the ortho-image and vertical error for the DEM. V G represents the error in the angular orientation of the candidate branch G of the graph, and V N the orientation of the velocity vector, a common navigation solution.

[0108] - HAS w corresponds to the orientation covariance sum of the navigation orientation error covariance Av directly calculated from the variance on the current velocity vector of the filter and the covariance A Ov describing the orientation error of the candidate edge of the graph. This is obtained in particular from the metadata of the geographic products used.

[0109] Among all the candidate associations, the simplest approach then selects the one with the lowest cost. To mitigate potential association ambiguities, a more complex approach maintains a number of possible association hypotheses over time, making final assignments to the graph data as soon as a doubt is resolved at a distance / orientation threshold. If doubt persists, the system can raise an alarm.

[0110] Preferably, the matching is also a function of the digital terrain model of the navigation area, stored in memory 22.

[0111] The determination step 120 also includes the determination of a set of corrections to the current position and current orientation of the optronic system 18, to realign at least one identified element on the imaging data with the corresponding characteristic element of graph G.

[0112] In one implementation example, at least one transverse realignment correction is determined for the current position and orientation of the optronic system 18 relative to the identified element under consideration. Assuming that the optronic system 18 is moving along a track, this allows for realignment of the current position and orientation on the track, particularly in the event of a deviation off the track.

[0113] In one implementation example, when at least one identified element is a track intersection and / or a track with a specific curvature, the matching is also performed based on the shape of the identified element. The shape incorporates, for example, the curvatures of the characteristic element. The correction set includes at least one longitudinal realignment correction for the current position and orientation of the optronic system 18 relative to the identified element in question.

[0114] As an example of an application, on most of these routes a wheeled military vehicle will be traveling on asphalt, and the processing will determine its transverse position relative to the road based on the position of the road edges in the sensor image. In practice: Relatively speaking, the road in front of and behind the vehicle is always positioned around the same locations on two 180° branches in the panoramic image; this presence confirms the vehicle is traveling on a surface with the specified semantic segmentation class.

[0115] In absolute terms, we have an approximate orientation based on the branch of the graph G traversed a priori, as well as an absolute orientation based on the velocity vector of the current navigation solution. These two corresponding directions should be close, within errors.

[0116] When approaching a curve edge G with a very pronounced radius of curvature, it becomes relevant to monitor and estimate the curvature of the road edges in front of the vehicle. This mechanism provides longitudinal alignment correction to the road, except in the presence of lane intersections.

[0117] NAVIGATION UPDATE AND MAINTENANCE

[0118] The process includes a step 130 of updating and maintaining the navigation of the navigation device 20 according to the set of corrections determined.

[0119] Navigation is maintained automatically, i.e. without requiring intervention from the crew or occupants of vehicle 19.

[0120] The update consists, for example, of correcting the position and orientation of the optronic device according to the set of corrections determined (angular deviation correction in particular).

[0121] The quality of navigation is indicated by the state covariance of the filter.

[0122] As an optional addition, the navigation filter is also powered by correspondences between directions provided by the imagery data using: relative observations between images that can be associated using classic techniques to construct image correspondences usable according to classic approaches such as visual odometry, Motion From Structure (MFS), or Simultaneous Localization and Mapping (SLAM) to feed the filter with spatial direction correspondences on common fixed objects in the scene; and absolute observations with a panoramic view at a sufficiently high site to acquire images of celestial bodies such as the moon or the sun, including their ephemerides.

[0123] For example, when the process detects a distance from the edges of graph G where its information becomes unusable, the re-registering is then limited to the use of celestial bodies or odometric measurements of the video to best maintain the navigation filter fed by IMU measurements. In this situation, the convergence of graph G is also detected in order to resume transverse and longitudinal track re-registering.

[0124] Thus, the default operation uses horizontal and vertical position corrections. The vehicle's altitude can also be estimated based on its height above the road and the road's altitude obtained from the horizontal coordinates of a DEM (Digital Elevation Model) if available, or more generally by interpolating the vertical information carried by the graph's nodes.

[0125] Updating and improving a position in a vehicle's inertial navigation system also improves the vehicle's attitude calculation. Improving this attitude information at high rates facilitates association decisions by comparing the orientation contribution to those of the graph segments.

[0126] Optionally, an additional phase is added for consistency checking. This analyzes the consistency of the IMU data:

[0127] - Analysis / monitoring of inertial data and detection of jumps: o detection of sudden jump inconsistencies in kinematic data, o detection of stop and start by analyzing accelerations / velocities.

[0128] Exploitation of inertial measurements and consistency with graph G and image data: o of plane accelerations for turn detection and acceleration-radius of curvature consistency. o of longitudinal attitudes for consistency with terrain or consistency of vehicle tilt based on its attitude relative to the terrain.

[0129] Preferably, if no features are identified in the image data, the steps of determining a set of alignment corrections and updating and maintaining the navigation system are interrupted. These steps are resumed as soon as at least one feature is identified in the image data. In an implementation example, if a panoramic image does not reveal any lanes in front of or behind the vehicle, the vehicle is declared as moving outside the graph. This state allows the processing to be adapted and a procedure to be launched to evaluate and confirm its approach to the graph during future movements.

[0130] Thus, when off-road, the optronic system detects when the vehicle leaves the road and therefore the graph. In this case, it relies on the navigation device 20 (the inertial measurement unit) for navigation and predicts the next intersection with the graph (road or track crossing), allowing for realignment at that point. Following the navigation update and maintenance step, at least the corrected position of the optronic system 18 is stored in the memory 22 of the optronic system 18 for later reuse in case of an interruption during the steps of determining a set of realignment corrections and updating and maintaining the navigation. This can also be used for debriefing, evaluation, or fine-tuning phases during or after a mission.

[0131] ADVANTAGES AND CONCLUSION

[0132] The invention solves the problem of recalibrating the navigation of a military vehicle without GNSS, by using a semantic segmentation of an optronic vision of the scene and by appropriately recalibrating the information extracted from the images to a graph G of the onboard channel network and best representing the information that can be automatically extracted from the environment.

[0133] The crew is thus exempt from manually intervening every few tens of kilometers if it has a control unit with a drift of 1m / km and a setpoint of 10m maximum deviation, for example.

[0134] The proposed solution allows operation in structured environments, but also in unstructured environments; that is to say, including outside urban areas and outside of paved roads such as paths, tracks in desert environments, fields....

[0135] The vehicle is limited to passive sensor equipment for reasons of electromagnetic discretion and energy consumption. The usable sensors are limited to inertial components and an optronic sensor providing video of the vehicle's surroundings.

[0136] In particular, the process analyzes the graph G and dresses it with geometric properties (e.g. road width), topological properties (e.g. intersection with road junction having an L, O, T or X shape) and semantic properties (e.g. road, path, track).

[0137] In particular, during navigation the optronic sensor acquires a high frame rate video of the landscape and the images are subjected to semantic classification by means of another neural network allowing to infer classes of the environment and in particular classes such as roads, paths, grass, sidewalk.

[0138] Preferably, a high frequency (HF) analysis allows the vehicle's rolling area to be located and characterized in relation to the elements of graph G and the vehicle to be located transversely at high speed.

[0139] Preferably, a low-frequency (LF) analysis is requested at the junction approach or at relevant features to locate the vehicle longitudinally to the road / path. This sequencing is performed based on the local analysis of the graph G topology using the vehicle's position.

[0140] Image observations and the G-graph thus allow the determination of visual positions and orientations of the vehicle to correct and maintain the navigation filter. Those skilled in the art will understand that the embodiments described above can be combined to form new embodiments, provided they are technically compatible.

Claims

DEMANDS 1. A method for automatically recalibrating the navigation of an optronic system (18) operating within a navigation zone, the navigation zone comprising characteristic elements, the characteristic elements comprising channels, the optronic system (18) comprising the following elements integrated within the optronic system (18): a navigation device (20) capable of estimating over time at least one position and orientation for the optronic system (18) and of guiding the movement of the optronic system (18) according to the estimated position and orientation, an imaging device (21) capable of acquiring images of the environment as seen from the optronic system (18), a memory (22) in which is stored a graph (G) of the characteristic elements of the navigation zone as seen from the sky, a computing unit (28), the method being implemented by the optronic system (18) and comprising over time the following steps: estimation,by the navigation device (20), of a current position and current orientation for the optronic system (18), the acquisition of imaging data by the imaging device (21), the imaging data being an image or a stream of images of the environment seen from the optronic system (18), the processing of the imaging data by the computing unit (28), the processing step comprising the segmentation of the imaging data so as to highlight characteristic elements, called identified elements, on the imaging data, the determination, by the computing unit (28), of a set of recalibration corrections for the navigation device (20), the determination step comprising:, • the matching of at least one identified element on the imaging data with a characteristic element of the graph (G) as a function of the current position and current orientation of the optronic system (18), • the determination of a set of corrections to the current position and current orientation of the optronic system (18), to realign at least one identified element on the image data with the corresponding characteristic element of the graph (G), and the updating and maintenance of the navigation of the navigation device (20) according to the set of corrections determined.

2. Method according to claim 1, wherein during the step of determining a set of corrections, at least one transverse recalibration correction of the current position and orientation of the optronic system (18) relative to the identified element considered is determined.

3. Method according to claim 1 or 2, wherein the characteristic elements of the graph (G) include channel crossings and / or channels having specific curvatures, during the step of determining a set of corrections, when at least one identified element is a channel crossing and / or a channel having a specific curvature, the matching is also carried out according to the shape of the identified element, and the set of corrections includes at least one longitudinal recalibration correction of the current position and orientation of the optronic system (18) with respect to the identified element considered.

4. A method according to any one of claims 1 to 3, wherein during the step of determining a set of corrections, the matching includes searching for the characteristic elements of the graph (G), called candidate elements, within a predetermined perimeter around the current position, the element associated with the identified element being chosen from among the candidate elements according to a selection criterion.

5. A method according to claim 4, wherein the selection criterion stipulates that: the associated element is the candidate element closest to the current position, or in the presence of several candidate elements at a distance less than a predetermined distance from the current position, the associated element is the candidate element generating a lower association cost when implementing a probabilistic approach, preferably the probabilistic approach implementing a calculation of Mahalanobis distances between each candidate element and the current position.

6. A method according to any one of claims 1 to 5, wherein the graph (G) in memory (22) has been previously obtained by processing one or more ortho-images of the navigation area so as to highlight the characteristic elements of the navigation area on the ortho-image(s) and to extract the shape of said characteristic elements to form the graph (G).

7. Method according to claim 6, wherein the characteristic features are highlighted on the ortho-image(s) by a deep learning algorithm, for example an algorithm implemented by a neural network.

8. A method according to any one of claims 1 to 7, wherein a digital terrain model of the navigation area is also stored in the memory (22) of the optronic system (18), during the step of determining a set of corrections, the matching also being a function of the digital terrain model.

9. A method according to any one of claims 1 to 8, wherein in the absence of identified elements on the image data, the steps of determining a set of recalibration corrections, and of updating and maintaining the navigation are interrupted, said steps being resumed as soon as at least one element is identified on the image data.

10. A method according to any one of claims 1 to 9, wherein at the end of the navigation update and maintenance step, at least the corrected position of the optronic system (18) is stored in the memory (22) of the optronic system (18) for later reuse in the event of a possible interruption of the steps of determining a set of recalibration corrections, and of updating and maintaining the navigation.

11. Optronic system (18) evolving in a navigation zone, the navigation zone comprising characteristic elements, the characteristic elements comprising channels, the optronic system (18) being configured to implement a method according to any one of claims 1 to 10, the optronic system (18) comprising the following elements: a navigation device (20) adapted to estimate over time at least one position and orientation for the optronic system (18) and to guide the movement of the optronic system (18) according to the estimated position and orientation, an imaging device (21) adapted to acquire images of the environment seen from the optronic system (18), a memory (22) in which is stored a graph (G) of the characteristic elements of the navigation zone in view of the sky, and a computing unit (28).

12. Vehicle (19), such as a land vehicle, comprising an optronic system (18) according to claim 11.

Images