Synthetic Antennas for Improving Object Detection

Abstract

The present invention relates to a computer-implemented method, the method including the steps of receiving a series of distance measurements generated by a detection system that operates by emitting waves from a number of different positions, receiving the waves reflected by an environment, and determining the distance by calculating the difference between the time of emission of the waves and the time of reception of the reflected waves, generating (420) a composite image representing the distance from the environment relative to a reference position based on the series of distance measurements, generating a focused composite image at the focus distance from the series of distance measurements or the composite image by applying compensation for the penumbra effect for each of a number of focus distances, and detecting the presence of an object in the focused composite image.

Description

[Technical field]

[0001] The present invention relates to the field of object detection based on sensor signals, and more particularly to object detection in a composite antenna signal by combining distance measurements at different positions. [Background technology]

[0002] Sonar is a measurement device widely used in underwater navigation to detect / locate and measure distances to underwater objects. Active sonar works as follows: -Sound waves are emitted, -Sound waves are reflected by objects in the water and on the ocean floor, - The reflected sound waves are picked up at a location close to the point of emission. The time difference between emission and reception and the intensity of the reflected wave allow the distance from the nearest point (object or seabed) in a given direction to be measured. The distances in different directions are aggregated to form a sonar image, which represents the distance from the nearest point to the sonar in different directions.

[0003] An object in a sonar image can be detected using the shape of the object's echo in the image, but its shadow, i.e. the shape of the part of the seabed not reached by the sound waves emitted by the sonar, may also be used as it is masked by the object.

[0004] To improve the sonar resolution, so-called synthetic antenna sonar systems may be used. Synthetic antenna sonar aims to improve the resolution at a given range without increasing the physical linear dimensions of the receiving antenna. The principle of synthetic antenna sonar is to use a physical composite antenna formed by a linear array of N transducers. In this type of sonar, when the carrier wave is moving forward, an emitter, or radiating antenna, emits M successive pulses at a fixed basic interval relative to the carrier wave. The signals received by the N transducers of the physical receiving antenna at M time points, and therefore at M consecutive positions, are used to form a synthetic antenna beam. The resolution of the acquired image, i.e. the resolution of the synthetic antenna beam ("array beam resolution"), is substantially equal to the length of the virtual antenna corresponding to the length traveled by the physical antenna at these M consecutive time points.

[0005] Synthetic antenna sonar is widely used because it can significantly improve sonar resolution without requiring any hardware changes. Summary of the Invention [Problem to be solved by the invention]

[0006] However, the shadow of an object seen by a synthetic antenna sonar is subject to the penumbra effect, also known as the parallax effect: because the angle of wave emission and echo reception varies between image acquisitions, the direction of the shadow also varies from image acquisition to image acquisition, resulting in a blurred shadow when generating a synthetic image.

[0007] In some cases, for example when an object is elongated in height or suspended between two bodies of water, the shadow can become nearly undetectable. This is the case for example with schools of fish.

[0008] Similar problems may be encountered with other types of synthetic antennas, i.e. sensors that operate on the principle of emitting and receiving waves at different locations and generating a synthetic image based on the reflected waves received at these different locations, as is the case for example with synthetic aperture radar, or with certain types of ultrasound.

[0009] Therefore, there is a need to improve detection of shadow-cast objects affected by the penumbra effect using a composite antenna based on emitting waves and receiving waves reflected at various locations. [Means for solving the problem]

[0010] For the above purposes, one subject of the present invention is a computer-implemented method comprising the steps of receiving a series of distance measurements from a plurality of different locations generated by a detection system operating to emit waves, receive waves reflected by an environment, and determine distance by calculating the difference between the time of emission of the waves and the time of reception of the reflected waves; generating a composite image representing the distance of the environment from a reference location based on the series of distance measurements; generating, for each focus distance of a plurality of focus distances, a composite image focused at the focus distance by applying penumbra effect compensation based on the series of distance measurements or the composite image; and detecting the presence of an object in the focused composite image.

[0011] Advantageously, the detection system is a sonar system and the generation of the synthetic image defines a synthetic antenna sonar.

[0012] Advantageously, the step of detecting the presence of an object in the focused composite image comprises applying a supervised machine learning engine trained with a learning base comprising focused images of shadows of objects similar to the object.

[0013] Advantageously, the method includes, prior to detection, the steps of calculating, for each pixel of the focused composite image, a ratio between the intensity of the composite image pixel and the intensity of the composite image, thresholding pixels of the composite image where the ratio is greater than a threshold, applying a mathematical morphological operation to the thresholded pixels, and applying the detection to the output of the mathematical morphological operation.

[0014] As an advantageous feature, the multiple focus distances include multiple initial focus distances defined by a first distance pitch over a first range of focus distances, and the method includes a step of defining multiple refined focus distances, the refined focus distances surrounding a first focus distance among the multiple initial focus distances at which the presence of an object is detected and defined by a second focus distance range narrower than the first focus distance and a second distance pitch narrower than the first focus distance, and a step of generating, for each refined focus distance of the multiple refined focus distances, a synthetic image focused at the focus distance by applying penumbra effect compensation based on the series of distance measurements.

[0015] Advantageously, the step of generating a composite image focused at said focus distance by applying a penumbra effect compensation on the basis of said composite image is performed by applying a one-dimensional filter to the composite image.

[0016] As an advantageous feature, the method according to claim 1 comprises the steps of generating a modified focused composite image by adding a shadow associated with a label to the composite image focused at the focus distance, generating a modified focused composite image by applying the inverse filter of the one-dimensional filter to the modified focused composite image, and enriching a learning base for detecting the presence of an object with the modified focused composite image, the focus distance and the label.

[0017] Advantageously, the method includes the step of generating a composite image based on said composite image and said focused composite image if the presence of an object is detected in said focused composite image.

[0018] Advantageously, the step of generating the composite image comprises the steps of detecting a shadow in the focused composite image and, for each pixel of the composite image, assigning to each pixel belonging to the shadow the intensity value of the corresponding pixel in the focused composite image and to each pixel not belonging to the shadow the intensity value of the corresponding pixel in the composite image.

[0019] As an advantageous feature, the step of generating the composite image includes a step of assigning, for each pixel of the composite image, an intensity value equal to a weighted sum of the intensity value of the corresponding pixel of the focused composite image and the intensity value of the corresponding pixel of the composite image, wherein, for each pixel, the relative weight of the intensity value of the corresponding pixel of the focused composite image increases according to an indication of the degree to which it belongs to the shadow of that pixel.

[0020] Advantageously, the method includes the steps of defining a region of interest of a composite image, generating a focused composite image at said focus distance by applying penumbra effect compensation based on said series of distance measurements, and detecting the presence of an object in said focused composite image, said steps being performed only within the region of interest.

[0021] Advantageously, selecting a region of interest in a composite image comprises the steps of displaying the composite image on a graphical interface and a user drawing a rectangle defining the region of interest.

[0022] Advantageously, the method includes the step of displaying a focused or composite image within said rectangle.

[0023] Another subject of the invention is a computer program product comprising computer code instructions, which, when executed on a computer, cause said computer to carry out the method according to one of the embodiments of the invention.

[0024] Another subject of the invention is a data processing system including a processor configured to carry out the method according to one of the embodiments of the invention.

[0025] Another subject of the invention is a computer-readable recording medium comprising instructions which, when executed by a computer, cause said computer to carry out a method according to one of the embodiments of the invention.

[0026] Other characteristics, details and advantages of the invention will appear on reading the following description and with reference to the accompanying drawings, given by way of example, in which: [Brief description of the drawings]

[0027] [Figure 1] 1 illustrates an example of generating a flow of synthetic antenna sonar images that can serve as the basis for object detection in one set of implementation modes of the present invention. [Figure 2a] 1 shows a first example of a shadow in a sonar image that can be used with a method according to one set of embodiments of the present invention. [Figure 2b] 4 shows a second example of a shadow in a sonar image that can be used with methods according to one set of embodiments of the present invention. [Diagram 3] 1 illustrates an example of a penumbra effect that can be compensated for as part of an embodiment of the present invention. [Figure 4a] 1 illustrates a first example of a computer-implemented method in one set of embodiments of the present invention. [Figure 4b] 1 illustrates a second example of a method according to one set of embodiments of the present invention. [Diagram 5] 1 illustrates one exemplary application of penumbra effect compensation in one set of embodiments of the present invention. [Figure 6] 1 illustrates an example of a graphical interface for defining a region of interest and for displaying a focused composite or composite image within the region of interest. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] The invention will be illustrated with examples relating to object detection using synthetic antenna sonar, but the invention is more generally applicable to any type of synthetic antenna based on emitting waves at various locations, receiving the reflected waves, and generating a synthetic image based on the reflected waves.

[0029] FIG. 1 illustrates an example of generating a flow of synthetic antenna sonar images that can serve as a basis for detecting objects in one set of implementation modes of the present invention.

[0030] In this example, a vehicle-mounted sonar S moves along an arbitrary, approximately linear trajectory above the ocean floor. At regular intervals, S emits pulses to image the background along a line of sight generally directed toward the edge of the trajectory.

number

number

number

[0031] The pulse is represented by a complex function p that is a function of either the pulse's travel time t or the sonar range r at the time of emission, the two variables being related by r=c×t, where c is the speed of sound. The pulse is a narrowband modulated carrier of wavelength λ.

[0032] The principle of synthetic antenna sonar operation,

number

[0033] The sonar S is formed by a radiating antenna and a receiving antenna. The radiating antenna is arranged to radiate energy in an antenna lobe. For example, lobes 141, 142 represent antenna lobes of radiating antennas radiating pulses with indexes n-1, n-2, respectively. The antenna lobe has an aperture β in the horizontal plane and an aperture with an azimuth large enough to illuminate a wide area of ​​the bottom. If the lobe is large enough, object A is visible with several successive pulses, and the energy is maximized when the location is exactly in the line of sight of the sonar.

[0034] Assuming that A is the only target, the raw acoustic signal received by the sonar at the nth pulse at range r can be modeled as:

number

[0035] Here, -j is an imaginary number, -K is the binding budget term related to the sonar-to-target distance and the target characteristics, -G is the antenna and vector

number

number

number

[0036] In reality, the scene consists of many targets 160. As the pulses progress, the synthetic antenna integration process is as follows: At emitted pulse number n, pitch δr* on the axis perpendicular to the trajectory is

number

number

number

[0037] Here is max is chosen for maximum sonar range.

[0038] With index k fixed, the position {P n (s, k)} forms what is called the kth beam of pulse n (or the kth beam of ping n) 160, and the coordinates k and s are called the beam index and sample index, respectively. n {(s,k)} is the assumed reference target.

[0039] Then, for a subset of beams, K n ={k n,min k n,max}, but the positions of these beams are chosen such that the gain at pulse n (with respect to the sonar position and attitude at that pulse) is greater than the gain from the sonar position and attitude at any other pulse obtained at the same point in space (in other words, these are the positions that can be best imaged at pulse n).

[0040] Each position (s,k), k∈K n In contrast, P n The pulse interval I(n,s,k) during which (s,k) is significantly illuminated by the sonar is determined as follows:

number

[0041] Next position

number

number

[0042] Beam K n ={k n,min k n,max}Data S SAS (n,s,k) is called the SAS antenna for pulse n, where SAS is an abbreviation for synthetic-aperture sonar. Equation 4 is known as the generalized backpropagation equation.

[0043] Next, we use a function called the waterfall function, W SAS Let (i,b) be defined as follows:

number

[0044] Here, the notation |K m | is a set K m The waterfall can be considered as a complex two-dimensional image, with the axis of beam index b called the azimuthal axis and the axis of index i called the radial axis. It is also possible to parameterize the waterfall in terms of the oblique distance r=s×δr and the curvilinear coordinate x=b×δx.

[0045] FIG. 2a shows a first example of a shadow in a sonar image that can be used with methods according to one set of embodiments of the present invention.

[0046] FIG. 2b shows a second example of a shadow in a sonar image that can be used in a method according to one set of embodiments of the present invention.

[0047] Shadows are an important factor in SAS images that contribute to the understanding of the scene by the operator or to automatic object detection. In fact, due to the very nature of the imaging process, echoes are arranged relatively compactly on an oblique distance axis, which greatly hinders shape recognition by human operators or machine algorithms. Conversely, if the shadows are elongated, it becomes possible to infer the shape of the object.

[0048] 2a and 2b show two examples of shadow recognition in sonar images, showing two scenes 210a, 210b on the seabed 211 and two sonar images 220a, 200b of these two scenes, respectively.

[0049] The two scenes each show an object 212a resting on the ocean floor and a similarly sized object 212b suspended between the two bodies of water and attached to the ocean floor by a cable 213b (rope) attached to a base 214b (sinker). The images are generated by a sonar located at location 230.

[0050] For scene 210a, the sonar detects echo 221a and shadow 222a of object 212a. For scene 210b, the sonar detects echo 221b of object 212b, cable 213b, and base 214b, as well as shadow 222b of object 212b and cable 213b.

[0051] A shadow is therefore defined as an area where no echo is perceived because it is masked by an object. As can be seen in Figures 2a and 2b, for an object of the same size floating between two bodies of water (mid-water), the shadow is significantly larger in size.

[0052] FIG. 3 illustrates an example of the penumbra effect that can be compensated for as part of one embodiment of the present invention.

[0053] The penumbra effect, or parallax effect, is the effect obtained when a target is illuminated for only a portion of the synthetic antenna sonar image acquisition.

[0054] For example, in the case of FIG. 3, target A may be masked by object T. for example, - At iteration n-2, the sonar detects the position

number

number

number

number

[0055] Target A is therefore illuminated by sonar in iterations n-2 and n+1, but not in iterations n-1 or n. Figure 3 also shows that the shadow cast by object T varies as a function of image acquisition angle. In practice, this means that in synthetic antenna integration the shadow will be blurred rather than sharp, and will suffer from the penumbra effect.

[0056] While the penumbra effect described above may still be tolerable for objects close to the ocean floor, such as object 212a in Figure 2a, it can become problematic for long vertical objects, such as object 212b in Figure 2b, or objects floating between two bodies of water (floating in mid-water or near the surface), where the shadow seen by the synthetic antenna sonar may be too blurred or distorted to be recognized.

[0057] FIG. 4a illustrates a first example of a computer-implemented method in one set of embodiments of the present invention.

[0058] Method 400a is a computer-implemented method directed to detecting objects that experience a penumbra effect in measurements with a composite antenna.

[0059] The method 400a includes a first step 410 of receiving a series of distance measurements from a plurality of different locations produced by a synthetic antenna location system that operates by emitting a wave, receiving the wave reflected by the environment, and determining distance by calculating the difference between the time the wave is emitted and the time the reflected wave is received.

[0060] The method 400a is therefore applicable to any synthetic antenna based on emitting waves and receiving reflected waves at various locations, for example, synthetic antenna sonar, synthetic antenna radar, or scanner.

[0061] In one set of embodiments of the invention, the detection system is a sonar system forming a synthetic antenna sonar.

[0062] The method 400a then includes generating 420 a synthetic image representing the distance of the environment from a reference position based on the series of distance measurements.

[0063] This step involves forming a composite image of distance for a given location, e.g., in the example shown in Figure 1, measurements made at times n-2, n-1, n, n+1, and n+2 can be integrated to generate a sonar image of location 110 at time n.

[0064] This composite image is referred to simply as the "composite image," in contrast to the "focused composite image" introduced below in this specification, and may also be referred to as the "reference composite image."

[0065] Method 400a then includes, for each focus distance of the plurality of focus distances: - generating 430 a composite image focused at said focus distance by applying penumbra effect compensation based on said sequence of distance measurements or on said composite image; - detecting 440 the presence of an object in the focused composite image.

[0066] In other words, at each distance in a given set of focus distances, the method 400a generates, in step 430, a focus composite image at the desired distance and then detects, in step 440, the presence of an object in the focus composite image.

[0067] Step 430 allows obtaining a synthetic image with clear shadows at the investigated distance, thus improving the accuracy of object detection since object detection in step 440 can be performed based on an image with clear shadows at a given distance.

[0068] Steps 430, 440 may be repeated for each of the multiple distances, so that for all desired distances, a clear shadow is available for detection.

[0069] The method 400a can thus improve object detection in a composite image from a composite antenna that is subject to the penumbra effect.

[0070] In step 430, various penumbra effect compensation methods can be used that have the effect of producing sharp shadows at a given distance.

[0071] For example, compensation for the penumbra effect is performed using a method known as "FFSE" (fixed focus shadow enhancement), described for example in Groen, J., Hansen, RE, Callow, HJ, Sabel, JC, & Sabo, TO (2008) "Shadow enhancement in synthetic aperture sonar using fixed focusing", IEEE journal of oceanic engineering, 34(3), 269-284. The method is presented in the paper for the case of an underwater vehicle moving on a straight trajectory with the sonar aimed at an angle of 90° from the trajectory. In this case, applying FFSE reduces the distance from the object r t The integral equation of the synthetic antenna sonar is replaced with the following equation:

number

[0072] In this case, position P n The image associated with (s,k) is blurred, but the transition between the image and the shadow is sharp. This operation maps all the positions in the range that are farther away to a distance r T The waterfall or synthetic image associated with an FFSE image has the same precision (i.e. number of pixels and resolution) as the synthetic antenna sonar waterfall or synthetic image.

[0073] Another possible penumbra compensation method is known as the HVPC (Higher Order Phase Compensation) method. The HVPC algorithm can be conceptually seen as an improvement of FFSE that takes into account the height of the shadow-causing object. This height may be determined in various ways, for example using an interferometer or by a third-party sonar such as a volume sonar. The height of the shadow-causing object can also be obtained by triangulation based on the length of the shadow cast by the object on the background, and by knowing the sonar altitude (and possibly the local water depth determined using an interferometer or by a dedicated sonar).

[0074] FIG. 5 illustrates one exemplary application of penumbra effect compensation in one set of embodiments of the present invention.

[0075] Two images of an underwater scene depicting the remains of a shipwreck are shown in Figure 5.

[0076] On the left, an image 510 is acquired using synthetic antenna sonar. In the image, debris 511 and the shadow 512 cast by the debris are blurred.

[0077] On the right, image 520 is acquired using the same synthetic antenna sonar and benefits from the application of a penumbra compensation method, in this case also FFSE, parameterized by the distance of the wreckage from the reference point of image acquisition. In the image, the wreckage 521 and the shadow cast by the wreckage 522 are clearly visible.

[0078] This example illustrates the ability of the penumbra effect compensation method in one set of embodiments of the present invention to produce sharper shadows at a given obstacle distance. The shadows thus obtained are sharper and therefore may be used to perform object detection more efficiently.

[0079] According to various embodiments of the present invention, the multiple focus distances can be obtained in various ways. For example, multiple predefined distances can be used. These distances can be defined by applying a distance pitch over a given range. The distance pitch can be defined such that sufficiently sharp shadows are obtained at all possible distances within the range.

[0080] The focal distance is, for example, within a certain distance range [r min ;r max ]. That is, the focusing distance may be defined as the distance equally spaced over the set {r min ;r min +Δ1;r min +2*Δ1···r max ]. The parameter r min ,r max and Δ may be defined in a variety of ways. For example, in one set of embodiments:

number

number

[0081] r min andr max The value of r may also be defined as a function of the minimum and maximum expected height of the object and the minimum and maximum expected distance. In fact, the minimum and maximum height and distance make it possible to identify the minimum and maximum distance of the shadow cast by the object. min andr max The values ​​of may thus be defined as the minimum and maximum focal distances at which a shadow is expected to be discerned in the remaining sonar range.

[0082] More generally, the value r min and r max can be defined to determine the minimum and maximum focal distances associated with searching for a cast shadow, thus restricting the calculation to distances where the shadow is discernible. For example, r max may be defined as the minimum of the maximum sonar range and maximum focusing distance at which a shadow is expected to be identified with the remaining sonar range.

[0083] These values ​​provide a good compromise between the objectives of limiting the number of focal distances to be tested (and therefore the computational complexity) and the objective of detection efficiency. min corresponds to the distance below which the parallax effect is not significant and the object can be detected directly without processing, and r max is the maximum detection distance of the sonar, and detecting a shadow beyond that distance is meaningless.

number

[0084] According to various embodiments of the present invention, the step of detecting 440 the presence of an object in the focused composite image may be performed in various ways.

[0085] In general, any shadow shape detection method may be used, for example machine learning methods may be used, which may include the use of artificial neural networks and / or deep learning.

[0086] In one set of embodiments of the invention, detecting the presence of an object in the focused composite image includes applying a supervised machine learning engine trained with a learning base that includes focused images of shadows of objects similar to the object.

[0087] The supervised machine learning engine may thus be trained using a training base formed of focused shadows of desired objects, for example, if the goal of the method is to detect the presence of a school of fish, the supervised machine learning engine may be trained using a database of focused images of the shadows of schools of fish.

[0088] Such supervised machine learning engines have the advantage that they can be trained to detect any kind of object and can do so very efficiently.

[0089] In one set of embodiments of the invention, the method comprises: -Prior to detection, For each pixel in the focused composite image, the ratio of the intensities of the pixel in the composite image to the pixel in the focused composite image I r Calculating (b,s); ·Related ratio I r thresholding pixels of the composite image where (b,s) exceeds a threshold; performing mathematical morphological operations on the thresholded pixels; - applying said detection to the output of said mathematical morphology operation.

[0090] If a pixel belongs to a shadow at the focus distance, it will be dark in the focus image, and therefore its intensity value in the focus composite image will be lower than its intensity value in the composite image. Therefore, the ratio of the intensity value in the composite image divided by the intensity value in the focus composite image will be greater than 1. The pixel is selected as a potential shadow pixel before applying the mathematical morphology. However, in one set of embodiments of the present invention, the selected threshold value is greater than 1 to avoid generating false alarms.

[0091] In other words, for each focus distance, step 440 may include a pre-processing step prior to the detection itself, which includes first thresholding the pixels corresponding to the shadows and then retaining only the significant shadows. The detection itself, e.g. the application of a supervised machine learning engine, is therefore applied only to the pixels that belong to the significant shadows.

[0092] This allows for more robust detection by limiting detection to the most prominent shadows at a given focus distance.

[0093] If detection is performed, it may be performed in a variety of ways. Non-limiting examples include: - A warning may be issued, A synthetic image associated with the detection may be presented to the operator; - Images may be stored in the image bank, - etc. may be carried out.

[0094] More generally, automatic object detection may be used in many fields, and any action that can be associated with automatic image detection may be performed upon completion of the detection.

[0095] In one set of embodiments of the invention, a focused composite image is generated based directly on a series of distance measurements for each focus distance, e.g., in the case of applying FFSE to measurements from a sonar, this means that the FFSE technique is applied directly to the sensor signals forming a SAS beam (SAS beamforming) for each focus distance.

[0096] Although the above-mentioned solutions provide a solution for obtaining an in-focus image, they may prove to be costly in terms of computation time if in-focus composite images need to be obtained for a large number of focus distances.

[0097] In order to limit computational complexity, in one set of embodiments of the invention, the step of generating a synthetic image focused at the focus distance based on the synthetic image by applying penumbra effect compensation is performed by applying one-dimensional filtering to the synthetic image.

[0098] The expensive step of generating a composite image based on distance measurements is therefore performed only once, and then all in-focus composite images are obtained by one-dimensional filtering, which is much less resource-intensive.

[0099] The method for computing the above-mentioned focused composite image is described below through an exemplary application to sonar imagery using FFSE.

[0100] The SAS waterfall w obtained using SAS processing sampled with pitch δx (finer than the physical resolution of the synthetic antenna) on the azimuth axis and pitch δr on the distal axis for the case where the sonar orientation is 90° to the straight trajectory. SAS Given (b,s), by 1D (one-dimensional) matched filtering (i.e. 1D correlation) based on the waterfall, a FFSE image can be obtained within a certain multiplicative factor, with the following signal described in the spatial domain and centered at b=0 for each column of constant index s:

number

[0101] Here,

number

number

[0102] The purpose of the above apodization function is to restrict the spatial support of the filter to a spatial region where the target is illuminated with a range s.δr, and the illumination has a typical standard deviation s.δr.

number

number

number

number

number

[0103] FFSE can then be applied to obtain a focused composite image by 1D filtering of the waterfall, which can be performed through 1D correlation of each column s.

number

[0104] In an equivalent manner, the above operations are performed by products in the spectral domain.

number

[0105] Here

number

number

number

[0106] Refocusing the SAS beam, i.e., transforming the composite image into a focused composite image, is done by -Image beam w SAS Fourier transform of (b,s)

number

number

[0107] Refocusing can thus be applied to an already formed SAS image and can be efficiently performed by a Fast Fourier Transform (FFT). Focusing can therefore be performed quickly and inexpensively over multiple focus distances.

[0108] Inverse Filter

number

number

[0109] In one set of embodiments, the above operations can be used to enrich a training database for object detection.

[0110] To this end, in one set of embodiments of the invention, method 400 comprises: - generating a modified focused composite image by adding a shadow associated with a label to said composite image focused at said focus distance; - generating a modified composite image by applying an inverse of the one-dimensional filter to the modified focused composite image; - enriching a learning base for detecting the presence of objects using the modified focused composite image, the focused distances and the labels.

[0111] In other words, a sharp shadow corresponding to a known object is added to the focused image at the focus distance, and then inverse 1D filtering is applied to obtain a reference synthetic image (e.g., a reference SAS image) containing the unfocused shadow. This image, together with the object label (defining the type of object) and the focus distance, is added to the training base for detecting the presence of the object. Thus, the combination of the modified synthetic image, the focus distance, and the label corresponding to the object type makes it possible to train object detection at various focus distances (then we know the focus distances at which the object type is likely to be detected or not detected).

[0112] This allows for the efficient construction of a training base, since it is possible to generate a large number of images at multiple focus distances, along with multiple shadow images for the training base, which allows for efficient and rapid training of object detection, for example by training a supervised learning engine for object detection, and greatly simplifies the construction of the training base, since it reduces the need to acquire real images.

[0113] FIG. 4b illustrates a second example of a method according to one set of embodiments of the present invention.

[0114] Method 400b includes all steps of method 400a, as well as three optional steps 450b, 460b, and 470b. Although these three steps are shown in FIG. 4b, it should be noted that these three steps 450b, 460b, and 470b may be performed independently. According to various embodiments of the invention, a method according to the invention may thus include these three steps, none of them, or any combination of these three steps.

[0115] In step 460b, the focus distance is tested more finely, ie, more finely around distances considered to be relevant.

[0116] In one set of embodiments of the invention, the plurality of focus distances thus includes a plurality of initial focus distances defined by a first distance pitch over a first range of focus distances, and the method further comprises: - a step 460b of defining a plurality of refined focus distances, said refined focus distances being: a second range of focus distances surrounding a first range of focus distances at which the presence of an object is detected among the plurality of initial focus distances and narrower than the first range; a step defined by a second distance pitch smaller than the first distance pitch; for each of the refined focus distances of the plurality of refined focus distances, generating 430 a composite image focused at the focus distance by applying penumbra effect compensation based on the series of distance measurements;

[0117] In other words, the initial focusing distance is the coarse pitch Δ1 (i.e., the initial distance is the set {r min ;r min +Δ1;r min +2*Δ1···r max}) with a range [r min ;r max ] may be a set of distances of the initial focus distance r festis selected from a number of focusing distances, and then an initial focusing distance r at which the presence of an object is detected with a finer pitch Δ2<Δ1 is selected. fest A refined distance range is defined around . For example, the refined focusing distance is in the interval [r fest -Δ1;r fest +Δ1] at a pitch Δ2.

[0118] A focusing step 430 is then performed at each refined focus distance.

[0119] This allows the focus distance to be tested at a finer pitch around the focus distance where the presence of the subject is detected, thus allowing focusing at a distance potentially closer to the subject distance.

[0120] This allows accurate results to be obtained while at the same time limiting the computational load required by the method since focus distances are tested at a coarse pitch throughout the entire possible range, but at a finer pitch in the range of focus distances of interest.

[0121] The first focus distance is therefore the focus distance at which the presence of an object is detected.

[0122] In other words, as soon as an object is detected in step 440, the focus distance becomes more refined around the focus distance that led to the detection of the object.

[0123] This allows for more refinement of object detection around the ranges that cause detection and therefore around the ranges that are most relevant for object detection.

[0124] In this case, the refinement is sometimes called auto-fixed focus, since it automatically determines the focus distance that produces the sharpest shadows.

[0125] Various sharpness indices may be used. As a non-limiting example, A. Buffington, F. S. Crawford, R. A. Muller, A. J. Schwemin and R. C. Smits, “Correction of atmospheric distortion with an image-sharpening telescope”, Jour. Acoust. Soc. Am., Vol. 3, pp. 298-303, March 1977 may be used.

[0126] In one set of embodiments of the invention, method 400b includes generating 470b a composite image based on the composite image and the focused composite image if the presence of an object is detected in the focused composite image.

[0127] In fact, as mentioned above, the focused composite image allows to obtain clear echoes and shadows at a given focus distance, but other parts of the image are blurred. Conversely, the unfocused composite image is clearer throughout the image, but the penumbra effect occurs. The composite image generated based on the composite image and the focused composite image is therefore simultaneously clear throughout the image and does not suffer from the penumbra effect, but on the contrary, may produce clear shadows at the focus distance.

[0128] The composite step 470b thus allows for the acquisition of the most consistent image possible for presentation to the operator, with sharp shadows at a given focus distance, without sacrificing sharpness in other parts of the image.

[0129] The above steps may be performed in a variety of ways.

[0130] In one set of embodiments of the present invention, step 470b of generating a composite image comprises: - detecting a shadow in the focused composite image; -For each pixel of the composite image For each pixel in the shadow, we use the intensity value of the corresponding pixel in the focused composite image. assigning to each pixel not belonging to the shadow the intensity value of the corresponding pixel of the composite image.

[0131] In other words, a shadow is detected in the focused composite image, and the pixels of the composite image are -If the pixel belongs to the shadow, -Pixels that do not belong to the shadow are taken from the composite image.

[0132] This provides a simple and effective solution for obtaining composite images with sharp shadows without sacrificing the overall quality of the image.

[0133] Other ways of generating the composite image are also contemplated.

[0134] For example, in one set of embodiments of the invention, generating the composite image includes assigning, to each pixel of the composite image, an intensity value equal to a weighted sum of the intensity value of the corresponding pixel of the focused composite image and the intensity value of the corresponding pixel of the composite image, wherein, for each pixel, the relative weight of the intensity value of the corresponding pixel of the focused composite image increases in accordance with an indication of the degree to which that pixel belongs to the shadow.

[0135] Thus, in this case, the intensity value of a pixel is defined as a weighted average of the intensity values ​​of both images, rather than being selected exclusively from one or the other image, and the relative weight of the intensity of the corresponding pixel in the focused composite image is greater if that pixel is considered to belong to the shadow.

[0136] This provides the benefit of shadow sharpness from a focused composite image, while at the same time providing the benefit of a more gradual transition between the shadow and the rest of the image.

[0137] The relative weights of pixels from the two images (composite image and focused composite image) can be calculated in various ways. For example, a low-pass filter may be applied to the focused composite image. In effect, the low-pass filter allows to efficiently determine whether a pixel forms part of a shadow or not.

[0138] For example, if we take a composite image, A Let (b,s) be the focused composite image. B Let (b,s) be the composite image, and let I c Let (b,s) be the focal distance under consideration, and let r T Then, the composite image is generated as follows: -The distance r from the definition T cannot handle shadows cast by objects in T closer than (s.δr <r T For every pixel that corresponds to a region, the value of that pixel is taken from the composite image: C (b,s)←I A (b,s) -Other pixels (i.e. s.δr ≥ r T The following processing is performed for each For all values ​​of b and s, I A Calculate the average of (b,s) a and I B Calculate the average of (b,s) b , Using a support filter with B beams and S samples (B and S are the filter tuning parameters), B Generation of low-pass filtered images of |I LP (m LP I LP (representing the average of For each pixel, the weight

number

number

[0139] Each pixel in the composite image is thus a weighted average of the corresponding pixels in the composite image and the focused composite image, giving more weight to the focused composite image in shadow regions and more weight to the composite image in other regions while ensuring a smooth transition between these two types of regions.

[0140] In one set of embodiments of the invention, method 400b includes a step 450b of defining a region of interest in the composite image, and generating a focused composite image 430 and detecting objects 440 are performed only within the region of interest.

[0141] This makes it possible to limit the complexity of the method, since steps 430, 440 are performed only on regions of interest that are deemed relevant.

[0142] In one set of embodiments of the invention, the region of interest includes the entire image based on the focus distance, while in other embodiments of the invention, the region of interest includes only a portion of the image, for example resulting from dividing the entire image into blocks of beams, or defined by a user.

[0143] FIG. 6 shows an example of a graphical interface for defining a region of interest and for displaying a focused composite or compound image in that region of interest.

[0144] In one set of embodiments of the invention, the step of defining a region of interest in the composite image comprises: - displaying the composite image in a graphical interface 610, 620; - the user draws a rectangle 621 that defines the region of interest.

[0145] FIG. 6 shows two successive states 610, 620 of such a graphical interface.

[0146] In the first state 610, the graphical interface represents a synthetic sonar image or waterfall. In this representation, the range is defined by the horizontal axis, with the right-most pixel corresponding to the location furthest from the sensor, and the beam is defined by the vertical axis, with the lowest location corresponding to the beam acquired on the earliest date and the highest location corresponding to the beam acquired on the latest date.

[0147] In the interface, the user can activate definition of an area of ​​interest by, for example, pressing button 630, and activate compositing by, for example, pressing button 631. However, these buttons are shown by way of example only, and other means may be used to activate definition of an area of ​​interest and / or compositing, such as keyboard shortcuts or voice commands.

[0148] In the example of FIG. 6, if button 630 is activated by the user, the graphical interface proceeds to step 620 and displays a rectangle 621 representing the area of ​​interest. The left vertical side of the rectangle 621 represents the shortest area covered by the region of interest. The right vertical side of the rectangle 621 represents the maximum extent covered by the region of interest. The upper horizontal side of the rectangle 621 represents the most recent beam covered by the region of interest. The bottom horizontal side of rectangle 621 represents the oldest beam covered by the region of interest.

[0149] In this example, the user can manually define the region of interest directly on the synthetic image. The user can change the region of interest by, for example, modifying the rectangle with a moving cursor (which can be manipulated using an input interface such as a mouse or a touch sensor). For example, the user can drag a corner, drag a side, or drag and drop the entire rectangle.

[0150] In some embodiments of the invention, method 400b includes displaying a focused composite or compound image inside the rectangle.

[0151] In the example of FIG. 6, a composite image is generated for a region of interest defined by rectangle 621 and displayed directly within that rectangle.

[0152] This allows the user to directly visualize the effect of focus on the image.

[0153] The composite image is generated and displayed when the user presses button 631 or executes another composite command (keyboard shortcut, voice command, etc.).

[0154] Alternatively, the decoding and display can be performed in real-time as soon as rectangle 621 is displayed and the user modifies rectangle 621. This allows the user to visualize the focusing and decoding results in real-time.

[0155] In the example of FIG. 6, focusing and shadow detection are controlled by a reference range r 2 represented by a line 622 that can also be moved by the user to change the parameterization. T Similar to the rectangle 621, the movement of the line 622 may trigger real-time refocusing, shadow detection, compositing, and display of the focused or composited composite image. T may be displayed (623).

[0156] The user can thus modify the shadow generation parameters and visualize the obtained results in real time.

[0157] The above examples illustrate the ability of the present invention to improve object detection in images from a composite antenna, but they are given by way of example only and are not intended to limit the scope of the invention as defined in the following claims.

Claims

1. 1. A computer-implemented method (400a, 400b) for detecting objects that are subject to a penumbra effect, the method comprising: receiving (410) a series of distance measurements from a plurality of different locations, the series being produced by a sonar detection system including a synthetic antenna, the detection system comprising: - Emits waves, - Receives waves reflected by the environment, - operates by determining distance by calculating the difference between the time of emission of the wave and the time of reception of the reflected wave, said method comprising: - generating (420) a synthetic image using said synthetic antenna based on said series of distance measurements, said synthetic image representing said distance of said environment from a reference position; For each focus distance of the plurality of focus distances, - generating (430) a composite image focused at the focus distance by applying penumbra effect compensation based on the series of distance measurements or the composite image; - detecting (440) the presence of an object in the focused composite image.

2. 2. The method of claim 1, wherein detecting the presence of an object in the focused composite image comprises applying a supervised machine learning engine trained with a learning base including focused images of shadows of objects similar to the object.

3. - prior to said detection, - calculating for each pixel of the focused composite image the ratio of the intensity of the pixel of the composite image to the intensity of the composite image; thresholding the pixels of the composite image where the ratio is greater than a threshold; - applying a mathematical morphological operation to the thresholded pixels; - applying said detection to the output of said mathematical morphology operation.

4. the plurality of focus distances include a plurality of initial focus distances defined by a first distance pitch over a first range of focus distances, and the method further comprises: - defining (460b) a plurality of refined focus distances, said refined focus distances being: a second range of focus distances that surrounds a first range of focus distances at which the presence of an object is detected and is narrower than the first range; - steps defined by a second distance pitch smaller than the first distance pitch; - for each of the plurality of refined focus distances, generating (430) a composite image focused at said focus distance by applying penumbra effect compensation based on said series of distance measurements.

5. The method of claim 1 , wherein the step of generating a composite image focused at the focus distance by applying penumbra effect compensation based on the composite image is performed by applying a one-dimensional filter to the composite image.

6. - generating a modified focused composite image by adding a shadow associated with a label to the focused composite image at the focus distance; generating a modified composite image by applying an inverse filter of the -1-dimensional filter to the modified focused composite image; - enriching a learning base for detecting the presence of an object with the modified focus composite image, the focus distance, and the label.

7. 2. The method of claim 1, further comprising the step of generating (470b) a composite image based on the composite image and the focused composite image if the presence of an object is detected in the focused composite image.

8. generating the composite image, - detecting shadows in the focused composite image; for each pixel of said composite image, For each pixel in the shadow, the intensity value of the corresponding pixel in the focused composite image is calculated. A method according to claim 7, comprising the step of: assigning to each pixel not belonging to a shadow the intensity value of the corresponding pixel of the composite image.

9. 8. The method of claim 7, wherein the step of generating the composite image includes the step of assigning to each pixel of the composite image an intensity value equal to a weighted sum of the intensity value of a corresponding pixel of the focused composite image and the intensity value of a corresponding pixel of the composite image, and wherein for each pixel, the relative weight of the intensity value of the corresponding pixel of the focused composite image increases according to an indication of the degree to which the pixel belongs to a shadow.

10. 2. The method of claim 1, comprising the step of defining a region of interest in the composite image (450b), - generating a composite image focused at said focus distance by applying penumbra effect compensation based on said series of distance measurements; and - detecting the presence of an object in the focused composite image, The method is performed only in the region of interest.

11. defining the region of interest in the composite image, - displaying said composite image in a graphical interface (610, 620); - a user drawing a rectangle (621) defining said region of interest.

12. 11. The method of claim 10 when dependent on claim 7, including the step of displaying the focused or composite image within a rectangle.

13. A computer program product comprising computer code instructions which, when said program is run on a computer, cause said computer to carry out the method of any one of claims 1 to 12.

14. A data processing system including a processor configured to perform the method of any one of claims 1 to 12.

15. A computer-readable medium comprising instructions that, when executed by a computer, cause the computer to carry out the method of any one of claims 1 to 12.

Images