A method for object recognition with increased representativeness.

Abstract

A method for recognizing an object of interest in a degraded two-dimensional digital image of the object. - pre-detecting objects of interest in a two-dimensional digital image and assigning labels to the objects (11); - reconstructing (13) the three-dimensional volume of this object of interest, thus labeled, from a plurality of available two-dimensional digital images (12) of this object; - storing in a database a record relating to this object thus reconstructed in three-dimensional form and labelled; -For each record thus stored, - generating (21) a plurality of new two-dimensional digital images according to a plurality of display modes from each of the three-dimensional volumes (14) thus reconstructed for each object; training (23) the neural network on a training set consisting of an extended set of two-dimensional digital images thus generated and corresponding to the labels of the objects of interest to be recognized (22); - from a degraded two-dimensional digital image of this object of interest to be recognized, - a step (30) of using the neural network thus trained to deliver as output the label of the object and a confidence index linked to the recognition of the object of interest; Includes.

Description

[Technical field]

[0001] The present invention relates to the recognition of objects in digital imaging. The invention applies generally to the automatic recognition of objects in digital images taken under difficult conditions, and in particular to the recognition of people from randomly taken two-dimensional (2D) digital images, or to the recognition of objects from digital images taken under difficult conditions (fog, long distances, object occlusion, exposure angles, low resolution images, etc.). [Background technology]

[0002] The field of artificial intelligence (AI) is currently experiencing exponential growth in many sectors. This growth is explained by the conjunction of three simultaneous factors: the development of learning algorithms, known as "machine and / or deep learning", the emergence of huge databases on the Internet ("big data"), and the increasing computational speed of the computers used to train the learning algorithms.

[0003] Generally, the recognition of objects in AI is based on training data sets, also simply called "datasets". Indeed, each training set contains input data that causes the generation of a model that supplies an output called the label of an image. For example, in supervised learning (classification), the output is known and the objective is to train the algorithm to respond to itself and deliver the labels of the objects thus recognized in the images to be processed.

[0004] It is known that in order to train a supervised learning algorithm, it is necessary to have a large amount of labeled input data. Currently, databases or banks of digital images typically rely on digital images of labeled objects, relatively limited variability in terms of exposure angles, and / or summarization (image transformation by rotating, shifting, adding / removing noise, etc.). Furthermore, the variability of human recognition is relatively limited (e.g., images of people to be recognized only from their faces). Summary of the Invention [Problem to be solved by the invention]

[0005] The aim of the present invention is to improve the situation, in particular by providing a solution which at least partially overcomes the abovementioned drawbacks. [Means for solving the problem]

[0006] For this purpose, the invention proposes a method for recognizing an object of interest in a degraded two-dimensional digital image of this object.

[0007] According to a general definition of the invention, the method comprises: - pre-detecting objects of interest in a two-dimensional digital image and assigning labels to the objects; - reconstructing a three-dimensional (3D) volume of said object of interest, thus labeled, from a number of available two-dimensional digital images of said object; - storing in a database at least one record of this object thus reconstructed and labelled in three-dimensional form; -For each record thus stored, - generating a plurality of new two-dimensional digital images from the thus reconstructed three-dimensional volume of each object according to a plurality of display modes, the exposure modes including exposure modes with different levels of occlusion and / or added noise; - training the neural network on a learning set composed of an extended set of two-dimensional digital images thus generated and corresponding to the labels of the objects of interest to be recognized; - from a degraded two-dimensional digital image of this object of interest to be recognized, - using the neural network thus trained to deliver as output the labels of the objects and a confidence index linked to the recognition of the object of interest; Includes.

[0008] Surprisingly, the applicant has found that generating a number of new two-dimensional digital images from a reconstructed three-dimensional volume of the object increases the representativeness (variability) of the training set (dataset) and therefore improves the confidence index of recognition with a two-dimensional image of the object to be recognized, even if the two-dimensional image is degraded.

[0009] According to preferred embodiments, the invention comprises one or more of the following features which can be used alone or in partial or full combination with each other: -If the confidence index exceeds a threshold, stop recognition, otherwise explore other elements to increase the success of the identification. As a non-limiting example, the three-dimensional volume reconstruction is a reflection tomography type reconstruction. As a non-limiting example, the size of the reconstructed three-dimensional volume of the object is 262x262x257 pixels. -Multiple 2D images obtained from a reconstructed 3D volume of an object belong to groups formed by 2D images at various angles (theta, phi, Phi, etc.), at different distances, with different occlusion rates, and with different noise. A number of two-dimensional images obtained from the reconstructed three-dimensional volume for an object of interest of the human type belong to a group formed by accessories such as hats, glasses, sunglasses and beards. As a non-limiting example, the resolution of the two-dimensional digital image thus produced is 124 pixels by 253 pixels. As a non-limiting example, the neural network is a convolutional neural network of the type ResNet50, ResNet101 or ResNet152 (residual network with 50, 101 or 152 neuron layers, respectively).

[0010] Furthermore, the invention relates to a computer program comprising program instructions for the execution of a method such as described above, when the program is executed on a computer.

[0011] Other characteristics and advantages of the invention will become apparent on reading the following description of preferred embodiments of the invention, given by way of example with reference to the accompanying drawings, in which:

[0012] Other advantages and features of the invention will become apparent from examination of the description and drawings. [Brief description of the drawings]

[0013] [Figure 1] 1 illustrates diagrammatically the main steps of the recognition method according to the invention; [Diagram 2] 2 represents diagrammatically the sub-steps of the step of configuration of a database according to the method according to the invention; [Diagram 3] Schematically represents an example of two-dimensional images of a ship taken in SWIR infrared (short wave infrared) on a horizontal plane from nine different angles for three-dimensional reconstruction of the object. [Figure 4] 2 represents diagrammatically the sub-steps of the step of generation of an extended database according to the method according to the invention; [Diagram 5] 4A-4C are schematic representations of example database records of 2D images for an object labeled "boat2E0A0" generated at various viewpoints and distances from a 3D volume of the object reconstructed from the 2D images of FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] With reference to Figures 1 to 5, we present three main steps for the automatic recognition of objects in challenging situations by an AI trained on a database of labeled images whose representativeness is increased through 3D reconstruction of the objects.

[0015] The aim of the first main step 10 is to construct a database of objects already identified and reconstructed in three dimensions.

[0016] Step 10 starts with sub-step 11 of preliminary detection of an already identified object of interest (e.g. a ship). Then there is rapid taking 12 of a limited but sufficient number of two-dimensional images (visible, infrared, active or passive) to perform a three-dimensional reconstruction of the object. Depending on the context of the object, two-dimensional images can be taken according to several scenarios (e.g. "ground-ground" scenario, "sea-sea" scenario, "air-ground" scenario and "air-sea" scenario). In the case of a ship, images can be taken according to scenarios such as "sea-sea" scenario and "air-sea" scenario. For example, with reference to FIG. 3, an example of two-dimensional images of a ship taken with SWIR on a horizontal plane ("sea-sea" scenario) from nine different angles for three-dimensional reconstruction of the object is represented.

[0017] From the two-dimensional images thus available (FIG. 3), a three-dimensional reconstruction 13 of the object is performed using an appropriate reconstruction method (e.g. reflectometry), and then a three-dimensional (3D) volume 14 of the object (in voxels) is obtained.

[0018] In fact, three-dimensional volumes can be obtained using reconstruction methods based on transmission or fluorescence (optical projection tomography, nuclear imaging or x-ray computed tomography), based on reflection (back reflection of laser waves) or based on solar reflection in the case of the visible band (0.4 μm-0.7 μm), or near infrared (0.7 μm-1 μm), or SWIR (1 μm-3 μm), or by considering the thermal radiation of the object (thermal imaging of 3 μm-5 μm and 8 μm-12 μm). This three-dimensional reconstruction process is described in the patent "Optronic system and method dedicated to identification for formulating three-dimensional images" (US Pat. No. 8,836,762 B2, EP Pat. No. 2,333,481 B1).

[0019] A set of voxels from a three-dimensional reconstruction with associated intensities is used, preferably the reconstruction having been obtained by back reflection.

[0020] At the end of the 3D reconstruction there is a database (i.e. {3Dvolume_Object(n)Label_Object(n)}, n=1,2,...,N (N is the number of records of identified objects)) containing records for the objects already identified.

[0021] It should be noted that the database can be enriched with objects from modeling or simulation.

[0022] The second main step 20 of the method according to the invention is to generate an extended database of 2D images in various configurations and to train a dedicated AI (Artificial Intelligence).

[0023] In fact, for each labeled object in the database, we generate a two-dimensional image that is taken from the three-dimensional volume thus reconstructed (21).

[0024] In one set of embodiments of the present invention, a three-dimensional volume is externally defined by a three-dimensional surface, and if the three-dimensional volume is incomplete, the three-dimensional surface is open.

[0025] For example, views taken from the three-dimensional volume are generated according to various angles (theta, phi, Phi) at different distances. In a set of embodiments of the present invention, the three-dimensional volume may also be modified, for example by application of different occlusion ratios and / or by using different additive noise.

[0026] In a set of embodiments of the present invention, adding noise or adding occlusions on the 3D surface results in a modification of the initial 3D surface to generate a new 2D image.

[0027] In the case of a face, the views obtained from the reconstructed 3D volume of the person to be identified can be of different types, with or without accessories (eg, hats, glasses, sunglasses, beard, etc.).

[0028] In a set of embodiments of the present invention, attachments are locally superimposed on elements of a 3D surface that can modify the 3D boundary of the reconstruction volume.

[0029] The multiple two-dimensional digital images thus generated according to multiple exposure modes from each modified or unmodified three-dimensional volume of the object are then associated with the object's label 22. Thus, multiple two-dimensional views corresponding to different viewpoints of the three-dimensional volume (and, if necessary, modifications of the three-dimensional volume) can be added to the training database.

[0030] Next, obtain the following elements: 3Dvolume_Object(n) → {2Dimage_Object(n,theta,phi,Phi,distance,Occlusion_rate,etc.),Label_Object(n)}.

[0031] Finally, for example, select a convolutional neural network of the residual network type (e.g., ResNet50) to be trained (23) on a training set thus generated and consisting of a set of two-dimensional digital images {2Dimages_Object(n)} corresponding to the labels {Labels_Object(n)} (n=1, 2, 3, ..., N for all objects N of interest).

[0032] The third major step 30 is to recognize the object of interest from the degraded 2D image of the object.

[0033] For example, preliminary detection of an ObjectX of interest consists of taking one or several two-dimensional images (visible, infrared, active or passive) in restrictive operating conditions (bad weather, long distance, object occlusion, any exposure angle, etc.).

[0034] The convolutional neural network thus trained is then used to deliver as output the label of the object of interest and a confidence index (score) linked to the recognition of the object of interest.

[0035] Provisions are made to stop recognition when the confidence index (score) is high (e.g., above 95%).

[0036] If the confidence (score) is low, the operator can explore other factors to increase the success of the identification.

[0037] As the database of already identified and reconstructed objects grows, the recognition confidence of the dedicated AI becomes stronger and, implicitly, the identification of any object becomes more successful.

[0038] As a non-limiting example, the recognition method was applied to a ship labeled "boat2E0A0" from a single 2D image generated from an exposure perpendicular to the surface of the ocean ("air-sea" scenario), which does not belong to the training 2D database. The image was resized to a resolution of 124 pixels by 253 pixels for compatibility with AI query processing.

Claims

1. 1. A method for recognizing an object of interest in a degraded two-dimensional digital image of the object, comprising: - a step (11) of pre-detecting said object of interest in a two-dimensional digital image and assigning a label to said object; - reconstructing (13) the three-dimensional volume of said object of interest thus labelled from a number of available two-dimensional digital images (12) of said object; - storing in a database records relating to said objects thus reconstructed and labelled in three-dimensional form; - for each record thus stored, - generating (21) from each of the thus reconstructed three-dimensional volumes (14) of the object a number of new two-dimensional digital images according to a number of display modes, the exposure modes including exposure modes with different levels of occlusion and / or additive noise; training (23) a neural network on a learning set composed of an extended set of two-dimensional digital images thus generated and corresponding to said labels of said objects of interest to be recognized (22); from a degraded two-dimensional digital image of said object of interest to be recognized, - using the neural network thus trained to deliver as output (30) separately the label of the object and a confidence index linked to the recognition of the object of interest; A method comprising:

2. The method of claim 1 , further comprising: stopping the recognition if the confidence index exceeds a threshold.

3. 3. Method according to claim 1 or 2, characterized in that the step (13) of reconstructing the three-dimensional volume of the object belongs to the group formed by reflection tomography and transmission tomography.

4. The method according to any one of claims 1 to 3, characterized in that the new plurality of two-dimensional digital images obtained from the reconstructed three-dimensional volume (14) of the object belong to groups formed by two-dimensional display mode images from the three-dimensional volume taken at various angles (theta, phi, phi, etc.), images taken at different distances, images with different occlusion rates, images with different noise.

5. The method according to any one of claims 1 to 4, characterized in that the new plurality of two-dimensional digital images obtained from the reconstructed three-dimensional volume (14) of an object of interest of the human type belong to a group in which an accessory is superimposed as a variable element on the new plurality of two-dimensional digital images.

6. The method according to any one of claims 1 to 5, wherein the neural network is a convolutional neural network of the type belonging to the group formed by ResNet50, ResNet101, ResNet152.

7. A computer program comprising program instructions for the execution of the method according to any one of claims 1 to 6, when the program is run on a computer.

Images