METHOD FOR RELATING A CANDIDATE IMAGE TO A REFERENCE IMAGE

Description

[0001] The present invention relates to the correlation of images, and more particularly to the correlation of at least a portion of a candidate image with at least one reference image, for the purpose of comparison or contrast with that reference image. The present invention relates more particularly to the technical field of image or point registration and the recognition of physical objects (for example, objects), notably for determining whether a physical object belongs to a predetermined class of physical objects. In practice, the present invention relates to a method for correlating at least a portion of a candidate image with at least one reference image. The present invention also relates to a method for generating a set of reference lists from a plurality of reference images, particularly for implementing the correlation method according to the invention.

[0002] Several image registration methods exist that directly compare two images of similar nature in order to align them or even superimpose them. Some of these widely used registration methods directly and independently identify points of interest and their associated local descriptors in the two images to be compared. They then attempt to successively match these local descriptors, and thus the corresponding points of interest, between the images. Finally, they search for an optimal geometric transformation linking these two images and, if necessary, apply it to one of them to achieve effective image registration.

[0003] Object recognition methods are also known, based on image analysis and direct comparison of the values ​​of elements in an image of a material subject with those in an image of a similar material subject (identical or of the same model), based on the global distribution of local attributes (texture, color, ...) matched between said images.

[0004] Thus, these existing methods, as disclosed for example in document FR2875628A1, first identify descriptors that adequately describe each image, then systematically and independently calculate, within each image, the points of interest corresponding to these descriptors. This necessitates a specific additional step of matching descriptors and points of interest, which is computationally intensive and detrimental in terms of time and resources.

[0005] These known registration and recognition methods therefore have limitations in their implementation, particularly in terms of efficiency, computation speed, and adaptability to specific cases. Furthermore, these methods poorly discriminate between relevant points of interest (in the sense of the desired objective) and irrelevant points; that is, they have a poor signal-to-noise ratio. Indeed, these methods are generally slow and resource-intensive, making them ill-suited for relating a set of disparate images, registering them, and / or comparing them for recognition or classification into predetermined categories. These methods remain very difficult to implement in real time on mobile devices such as smartphones when high-quality recognition performance is required.

[0006] There is therefore a real need to facilitate the linking of images and improve its performance to enable new applications. The present invention aims to achieve this objective.

[0007] Thus, the present invention relates to a method for relating at least a part of a candidate image with at least one reference image, comprising the following steps: a) implementation of a relational repository comprising at least: an ordered list of relational descriptors, at least one calculation method to be applied to images to determine descriptors of these images, and a method for determining the degree of similarity between two descriptors, b) implementation, for each reference image, of a reference list which includes the positions, called reference points of interest, in the reference image, of descriptors of the reference image similar to relational descriptors from a relational repository compatible with that implemented in step a), which reference list is ordered according to the order of this compatible relational repository, c) determination, in the candidate image, of descriptors of the candidate image calculated according to each descriptor calculation method of the relational repository implemented in step a),and determining the position of each of these descriptors in the candidate image, d) determining the degree of similarity, determined according to the method of determining the relational reference frame implemented in step a), between each descriptor of the candidate image and each relational descriptor of the relational reference frame, e) determining a candidate list that includes the positions, called candidate points of interest, in the candidate image, of the descriptors of the candidate image exhibiting the greatest similarity with the relational descriptors of the relational reference frame implemented in step a), which candidate list is ordered according to the order of this relational reference frame, f) processing the candidate list with respect to each reference list on the basis of the order of the candidate and reference lists.

[0008] It should be noted that, for the purposes of this invention, the steps are not necessarily performed in the sequential order listed: they may, in particular, be performed in reverse order or simultaneously. However, it is essential that step c) be performed before step d), which itself must be performed before step e), at least for each image descriptor. These steps may be performed simultaneously and in parallel for different image descriptors. Steps c) to e) are not necessarily consecutive, as intermediate steps may of course be inserted between steps c) to e), for example, step b). Finally, step f) must necessarily follow the creation of the candidate list provided for in step e) and the implementation of at least one reference list provided for in step b).

[0009] For the purposes of this invention, it is understood by " pictureWhether candidate or reference, an image can be any type of image in the general sense of the term, and not just an image comparable to a photograph. In other words, an image is not limited to an optical image resulting from the stimulation of the authentication region by visible light, but can instead be obtained by any type of physical stimulation, including but not limited to: ultrasound, far-infrared, terahertz, X-rays or gamma rays, X-ray or laser tomography, X-ray radiography, and magnetic resonance imaging. Thus, for the purposes of this invention, an image is, for example, the recording of the result of stimulation by any means of a natural scene or a material subject. This recording can then be described as a natural image.This recording may be one-dimensional, corresponding, for example, to the recording of the variation over time or along a line of a single signal, or to the recording of the values ​​from a line of sensors. This recording may also be two-dimensional, as is the case with a photograph, which can be recorded in halftones, grayscale, or color. For the purposes of this invention, an "image" can therefore be a 1D signal, a 2D or 3D image, in grayscale or color, or an nD (n-dimensional) signal, for example, hyperspectral or RGB-D. For the purposes of this invention, an "image" can be the result of a single acquisition or extracted from a stream. Thus, within the scope of this invention, an "image" can be extracted from a video stream. In one embodiment, the image can be saved digitally.Furthermore, an image used in the process according to the invention can be a pre-segmented image, particularly when it comprises several physical subjects in the same scene or a repetition of the same motif. Of course, an image is not necessarily natural and can be synthetic, meaning it can be generated by a computer process with or without the assistance of a human operator. Within the scope of the invention, a natural image and a synthetic image have in common that they are in the same digital or analog recording format for processing within the same process. Optical and / or digital image enhancement pre-processing can also be applied to it, for example, to improve the signal-to-noise ratio.Thus, it is possible to apply, for example, optical zoom (variable focal length devices) and / or digital zoom to better select the observation scale, image deconvolution to remove focus defects or camera shake, bandpass filtering to select / prioritize intermediate frequency details, or contrast enhancement to accentuate contrast.

[0010] An image is considered a "reference" image because it is known prior to the implementation of the process. The reference image is preferably taken from a photograph taken before the process was implemented. The reference image can also be a computer-generated image, that is, for example, an image constructed from images of physical subjects or scenes, or an image resulting from a purely computer-generated synthesis process, or even an image combining these two methods. The reference image is one of the parameters chosen a priori in the implementation of the process according to the invention. The reference image is recorded, and may even be certified, by a trusted third party.

[0011] An image is described as a "candidate" in that it is used to determine its degree of local relationship to a reference image. The candidate image is preferably a natural image, resulting from the acquisition of a signal generated by the stimulation of a physical object and / or a scene comprising one or more physical objects. The candidate image is generally, but not exclusively, acquired just before the implementation of the method according to the invention.

[0012] Preferably, the candidate image and the reference image(s) with which it is related (via the process according to the invention) are from the same type of physical stimulation (for example, stimulation by visible light radiation when the images are from a photograph of material subjects in globally diffuse visible light).

[0013] In the context of the invention, a " descriptorA descriptor is a computer or digital object, a data structure, that allows for the representation of certain local properties of an image. Thus, in its application to an image, a descriptor is associated with its location within the image, that is, the place or area of ​​the image that exhibits the characteristics defined by the descriptor or that correspond to the descriptor. Within the context of this invention, this location is also called a "position," salient point, or notable point. Descriptors can be of different types and / or forms. In the case of an image in the sense of a photograph or a two-dimensional color image, descriptors can be geometric in nature, for example, corners, vertical or horizontal lines, or letters; colorimetric in nature, for example, local maxima of the luminance gradient or local contrast; or spectral in nature, without this list being exhaustive.The descriptors can also be of various types, namely, they can be the result of a machine learning process or be constructed for implementing the invention's method in a specific context. A descriptor can take the form of a matrix (a vector, a patch, etc.) or a graph (planar or non-planar, an n-ary tree or not, etc.) or any other data structure, for example, a value or a number.

[0014] Within the scope of the invention, and by metonymy, a descriptor also refers to the result of a calculation method which, once applied to the constitutive data of an image, allows the description, via the data structure corresponding to the descriptor, of relevant predefined characteristics within said image. Examples of descriptor calculation methods are: HARRIS, SIFT, SURF, ORB, KAZE, RGB, VGG-16, which allow, for example, the search for corners, invariant elements with varied characteristics, and colors. A descriptor thus makes it possible to characterize any given point of an image and its neighborhood, based on chosen or learned characteristics, for example, based on texture or local contrast, the presence of a given shape, color, its intensity, or even the color gradient, the local distribution of the gradient orientation of an image component, or many other characteristics.An image descriptor calculation method, applied to the points of an image, allows the extraction of any notable points where the local features represented by the descriptor are intrinsically sufficiently present; in practice, these are the points where the degree of relevance of the descriptor resulting from the calculation method is sufficiently high. The set of notable or salient points usually constitutes an intrinsic representation of the image's content.

[0015] In practice, the method of calculating image descriptors also allows us to define the coordinates that are associated with the notable points of the descriptor in the image, in a chosen coordinate system which is, for example, in the context of two- or three-dimensional spaces, a Cartesian, polar or cylindrical coordinate system.

[0016] In the context of the invention, we mean by " referencerelational » a set of computer or digital objects enabling the linking of images, including at least: an ordered list of so-called "relational" descriptors; and for each relational descriptor in the ordered list, at least one method of calculation to be applied to a given image to determine descriptors of that image, and a method of determining the degree of similarity between said relational descriptor and each of the image descriptors associated with that relational descriptor.

[0017] The relational reference frame is one of the parameters chosen a priori (i.e. upstream) of the implementation of the process according to the invention.

[0018] Here, two types of descriptors are used: on the one hand, " relational descriptors » chosen and ordered from the list contained in the relational repository, and, on the other hand, from the « descriptors» reference and candidate images, which are the result of one or more calculation methods applied respectively to each of these images.

[0019] Relational descriptors can either be explicitly constructed ad-hoc or be the result of a calculation method applied to any image, distinct from the candidate and reference images. The relational descriptors in the ordered list of the relational repository can be of different natures and / or forms. In any case, the relational descriptors are known before the implementation of the method according to the invention.

[0020] The descriptors of the reference or candidate images are the results of the calculation method(s) associated with the relational descriptors. In practice, the descriptors most similar to the relational descriptors of the relational reference frame are selected from among these calculation results. This selection leads to the identification, in the corresponding reference or candidate image, of a set of positions, called "points of interest," which correspond to the notable points associated with the selected descriptors. For the purposes of this invention, a given notable or salient point of an image will therefore be called " point of interest"of this image, for this image descriptor, when the desired feature(s) associated with the descriptor possess(s) a sufficient degree of similarity with the feature(s) associated with one of the relational descriptors. The same point in an image can potentially be detected several times as a point of interest for different descriptors during the implementation of the method according to the invention. Thus, in the sense of the invention, a descriptor of a reference image or of the candidate image constitutes a means of detecting points of interest in these respective images, which points of interest are not intended to represent the entire content of the image but to contribute – through the n-tuples formed from their coordinates – to relating the candidate image in question to a reference image, via a relational frame."Where appropriate, such detection of points of interest can be carried out at several scales, at different image resolutions, and invariantly regardless of camera pose, illumination variations, or capture noise, during image acquisition.

[0021] Thus, for any relational descriptor considered in the ordered list of the relational reference system, the detection of a point of interest in the candidate image is achieved by applying, within that image, the calculation method associated with such a relational descriptor, in order to determine descriptors of the candidate image that are functions of the points of application of the calculation method within the image. In other words, for each relational descriptor constituting the ordered list of the relational reference system, the application of the calculation method associated with that relational descriptor, within an image, allows us, firstly, to determine whether that image contains at least one local and robust descriptor of that image that has a nature and form similar to that of the relational descriptor, and, secondly, to determine the position of that descriptor.Of course, it is entirely possible that the image presents several descriptors resulting from the calculation method associated with the relational descriptor, in which case several potential points of interest are detected.

[0022] When at least one descriptor results from the calculation method associated with the relational descriptor, this descriptor is called "transient." The similarity (or resemblance) between the relational descriptor and each transient descriptor obtained by the calculation is then compared, generally using a calculation. To this end, the invention provides for determining a degree of similarity between the relational and transient descriptors. Preferably, the degree of similarity between the descriptors must be greater than or equal to a minimum threshold set for the compared descriptors to be considered similar. Only when the descriptors are similar is the point retained as a point of interest.For the sake of simplicity and ease of terminology, we say that the relational descriptor "is found" in the analyzed image, at the point of interest, or that the point of interest "is associated" with this relational descriptor in the analyzed image. We can also, for example, classify transient descriptors according to this degree of similarity and retain several for the same relational descriptor, provided that we only retain those with a degree of similarity greater than a chosen threshold.

[0023] For example, one possible way to determine the degree of similarity between two descriptors is to measure a distance between the data structures representing each descriptor. In this case, the degree of similarity is the value of the measured distance, while the measure of the degree of similarity corresponds to the nature of the similarity calculation chosen, for example, calculating a distance in the sense of Hamming, Mahalanobis, Levenshtein, or Hausdorff.

[0024] For example, if the relational descriptor is a colorimetric descriptor describing a certain red gradient, the calculation method to be applied to the candidate image can be chosen as the RGB calculation method. If this calculation method results in a transient descriptor that itself provides a certain red gradient value at at least one point in the image, then the actual red gradient values ​​should be compared, for example by subtraction, and the result compared to a fixed threshold value to determine if the descriptors are similar.

[0025] In the rare cases where certain relational descriptors have no associated points of interest in a given image, due to an insufficient degree of similarity between relational and transient descriptors (according to a chosen threshold), or due to insufficient relevance of transient descriptors within a chosen area of ​​the image, specific processing must be applied to enable the creation and comparison of candidate and / or reference lists. This could be achieved, for example, by assigning a specific state to the relevant rank in the list. For the sake of simplicity, it is then said that the relational descriptor is not found in the analyzed image.

[0026] Thus, for each relational descriptor constituting the ordered list of the relational reference, the application of the corresponding real-valued degree of similarity measure between said relational descriptor and each of the transient descriptors calculated in the image considered, makes it possible to classify these transient descriptors with respect to each other in order to determine, if they exist, what the point(s) of interest is / are where the relational descriptor considered is found in accordance with the desired degree of similarity, in the analyzed image.

[0027] The points of interest obtained for the candidate image and each reference image are then stored in lists respectively called "candidate" or "reference." In other words, each of the reference lists implemented in step b) and the candidate list determined in step c) are obtained using the relational descriptors of the relational repository. A new candidate list is generally generated each time a new candidate image, as defined in the invention, is implemented.

[0028] In practice, each reference list and candidate list therefore includes the coordinates, in the corresponding image, of the points of interest where each of the relational descriptors of the relational repository is found. If a relational descriptor from the ordered list is not found in at least one of the images, no point of interest is associated with it in that image, so no coordinates are associated with it in the corresponding list. Conversely, if a relational descriptor is found in several points of interest in the image, the list will include each of these multiple points of interest. Preferably, the number of points of interest that can be associated with a given relational descriptor will be limited, for example to 1, 2, 3, or even more as needed.The points of interest associated with the transient descriptors exhibiting the highest degree of similarity with the relational descriptor in question will then be retained in the list.

[0029] According to the invention, "linking" images or parts of images means identifying, within each image or part of an image, the positions of points of interest according to predefined local characteristics (given by the relational descriptors of each relational frame), using methods for calculating descriptors and measures of degrees of similarity between descriptors, all of which together constitute each relational frame. It is therefore the relational frame (or frames of reference and knowledge of the relationship between them) that enables the linking of the candidate list to each of the reference lists, and consequently, the linking of the candidate image to each of the reference images. In the sense of the invention, image matching is thus a specific linking of images.In the context of the invention, the linking of a candidate image with a reference image is therefore carried out at the end of the determination of the candidate list and its comparison with the reference list, on the basis of the relational descriptors of the ordered list of the relational reference system.

[0030] Preferably, a candidate image is compared with several reference images, successively or in parallel.

[0031] The relational repository list is "ordered" in that we know the position (or rank) of each relational descriptor in the list. For example, we can assign a number to each relational descriptor based on its position in the list.

[0032] The order of each reference list and candidate list is determined by the order of the relational descriptor list in the relational repository. This means that, although the orders may differ between all these lists, the relationship for moving from one order to another is known. A special case, preferred here, is when the order of each reference list and candidate list is determined by the order of the relational descriptor list in the relational repository; that is, all the orders are identical, rank for rank.

[0033] Thus, the order of the relational descriptor list preferably remains unchanged for linking the candidate image with the reference images. Using the same ordered relational descriptor list to link two distinct images implicitly implies, in each of the two lists, the same order for the points of interest resulting from these images (an order which is constrained directly or indirectly by the said ordered relational descriptor list). This identical, or at least known, order facilitates linking the images through the possible matching of homologous points of interest between said images, or the determination of homologous data aggregates from said images, as will be explained below.The order is considered "indirectly constrained" for example when two distinct but compatible relational frames of reference are used to analyze the candidate image and the reference image, such that the order associated with the reference list and the order associated with the candidate list may be different, but the relationship that allows us to move from one order (that of the candidate list, for example) to the other (that of the reference list, for example) is known. This relationship exists in particular because the relational frames of reference are compatible with each other, as will be seen in the description below.On the contrary, the order is "directly constrained", for example when the same relational repository is used to establish each candidate and reference list, or when two distinct and compatible relational repositories are used, but the difference between these relational repositories is not related to the order of the relational descriptors they respectively contain.

[0034] Thus, thanks to the image linking process according to the invention, the determination of the reference lists and the candidate list, constrained by the choice of relational descriptors which are ordered in the list of the relational reference, allows a rapid sequential matching of homologous points of interest in these images.

[0035] The present invention therefore proposes a new approach to relating points and digital images prior to any subsequent image processing. This new approach improves the performance of existing methods used for image registration, object recognition, and classification among a large number of reference images.

[0036] The process according to the invention, in particular thanks to the relational reference system, allows both the linking of disparate images, which have no connection to each other and which contain very different content, for example an image representing a jewel with an image representing a landscape or a kitchen utensil, and the linking of more similar images, for example two models of products from the same brand.

[0037] The use of a relational database allows the method of the invention to achieve high computing speeds while requiring fewer computing resources. The method according to the invention therefore makes it possible to perform, in real time, including via mobile devices and video streams, a priori registration, recognition, and classification tasks.

[0038] The method according to the invention avoids using extreme computational methods (for example, massive brute force, on a large amount of data) by optimizing resources according to the chosen implementation, for example by partially using a part of the relational repository, in the first iteration, then a subsequent sequence using all or another part of said relational repository.

[0039] Preferably, the method according to the invention is implemented in such a way as to decouple the calculations of the reference lists from the calculation of the candidate list. Preferably, the reference lists are indeed generated before the method is applied to the candidate image, according to another method of the invention, which is a method for generating a set of reference lists according to the invention. In other words, preferably, the reference image has been analyzed from the perspective of the relational frame of reference before the method according to the invention is implemented, so that the corresponding reference list is known prior to the implementation of the method according to the invention and the analysis of the candidate image. Thus, in step b) of the relationship-making method according to the invention, it is sufficient to call the pre-established reference list, without needing to refer to the reference image itself.This limits the amount of information processed in real time by favoring the use of pre-saved data, which in this case are essentially reference lists made up of n-tuples of coordinates of points of interest calculated using the relational reference system. This results in additional time savings when linking the candidate image to this reference image, allowing for reduced calculations at this stage and a short response time.

[0040] The calculations according to the invention are therefore faster compared to the calculation times of already known methods. This advantage is further amplified when the method is implemented to analyze successive images from video sequences.

[0041] The following describe other advantageous features of the matching process according to the invention.

[0042] The method according to the invention uses a chosen relational reference frame, preferably autonomously or optimized with respect to the reference and candidate images. This optimization can be carried out automatically or manually, or even by combining the two approaches (speed and precision for the automatic part, the value of experience and meaning for the human part).

[0043] The relational framework is described as " autonomousThis occurs when the ordered list of relational descriptors it contains is chosen without prior analysis of the candidate image or each of the reference images. The autonomous relational repository allows the linking of two images to be decoupled from their respective content. The relational repository is described as an "indirect repository" when it predefines, without needing to know the reference and candidate images, the relational descriptors that will allow the identification and positioning of the local characteristics associated with said relational descriptors.

[0044] The relational reference frame is considered "optimized" when it is chosen based on an appropriate discriminatory power between reference images and / or for a given or expected family of candidate images. The relational reference frame can thus be easily modified or optimized with respect to the population of reference images or even the type of candidate image to be considered. This optimization is possible because, according to the invention, the list of relational descriptors of the relational reference frame does not aim to provide a relevant description of the overall informational content of each image taken individually, but rather aims to locate features defined a priori (upstream of the implementation of the process), in an optimized manner, by each relational descriptor; that is to say, to determine the corresponding point(s) of interest and ultimately an ordered set of coordinates associated with these points of interest.Thus, the process according to the invention makes it easy to combine the nature of the relational descriptors chosen a priori, and to adapt to the use case the possibility of recognizing a candidate image or even authenticating this image, the possibility of determining the membership of a candidate image in a family of images and by extension the possibility of authenticating a material subject present on the image and / or determining the membership of this material subject in a family or category of material subjects or even to perform these different tasks sequentially.

[0045] In particular, the relational framework can be optimized for a given task. It may differ depending on whether the task involves aligning images with each other, recognizing images or material subjects within images, comparing, or classifying images.

[0046] The relational descriptors of the relational repository can also be chosen according to criteria of repeatability and stability. When a repeatable relational repository is used multiple times to determine an ordered list from an image, the resulting list is the same each time, for example. Similarly, when the same stable relational repository is used multiple times to determine an ordered list from several different images that share common features, certain similarities emerge in the resulting lists, for example.

[0047] In one embodiment, the relational repository is optimized through a learning process. This learning process can be of any appropriate nature, for example, genetic algorithms or neural networks, to achieve the expected performance in the given use case. This learning process can be implemented automatically, with or without human intervention.

[0048] According to an advantageous feature of the method according to the invention, the ordered list of relational descriptors of the relational reference frame is optimized, at least with respect to the reference lists implemented in step b), so that the coordinates of the respective points of interest associated with each relational descriptor are different from one reference image to another. In other words, the ordered list is optimized to retain only the relational descriptors that best distinguish the reference images from one another, in that the relational descriptors are found at points of interest with different coordinates from one reference image to another.

[0049] According to another feature, the ordered relational descriptor list of the relational repository is optimized based on the reference lists implemented in step b), so that the distributions of the points of interest corresponding to each of the relational descriptors in the reference image are as far apart as possible from one reference image to another.

[0050] According to another feature, the ordered list of relational descriptors is optimized based on the reference lists implemented in step b), so that each point of interest associated with one of the relational descriptors in the reference image is locally distributed across each reference image. In practice, this means that the relational descriptors are each found at a single, distinct point of interest within the reference image.

[0051] According to an advantageous feature of the method according to the invention, the ordered list of relational descriptors of the relational repository is optimized with respect to at least the reference lists implemented in step b), so that the points of interest within each reference image are evenly distributed. "Evenly distributed" means that these points are randomly and relatively uniformly distributed across the different reference images.

[0052] According to the invention, it is perfectly conceivable that a first relational frame of reference be used to establish a reference list from a given reference image, and that another relational frame of reference, different from the first, be used to establish the candidate list from the candidate image, provided that these relational frames of reference remain compatible with each other so that switching from one relational frame of reference to the other is possible when relating the candidate image to the given reference image. Two relational frames of reference are considered compatible when they produce a similar list of points of interest, according to a chosen similarity criterion (or a chosen measure of degree of similarity), after having been used respectively with the same reference image.As explained previously, measuring the degree of similarity again involves, for example, calculating a distance (according to a chosen calculation method) between the lists and comparing the resulting value to a chosen threshold value. Alternatively, two relational reference systems are considered compatible when, firstly, the first relational reference system is used to determine a first and second list of points of interest from a first and second image, and the second reference system is used to determine a third and fourth list of points of interest from said first and second images, and secondly, a first relationship exists between the first and second lists of points of interest, and a relationship similar to the first relationship exists between the third and fourth lists of points of interest, according to a chosen similarity criterion.The relationship between lists can, for example, come from a statistical analysis and / or a geometric analysis of the lists.

[0053] Two relational reference frames implemented within the meaning of the invention (one to analyze the candidate image, the other to analyze a reference image) may, while being compatible with each other, differ, for example, in that: 1 / the ordered list of relational descriptors being identical in both relational repositories, the method of calculation and / or the measure of the degree of similarity are distinct for at least one relational descriptor, the measure of the degree of similarity remaining compatible with the method of calculation and said relational descriptor; or, 2 / the ordered lists of relational descriptors used in each relational repository are of the same cardinality and very similar (without being identical) with regard to the data structures of relational descriptors, term by term or taken as a whole; or, 3 / the ordered lists of relational descriptors used in each relational repository are of the same cardinality but very different term by term from the point of view of the data structures of relational descriptors, the order and ranks remaining compatible from one list to the other;or, 4 / the ordered lists of relational descriptors used in each relational repository are of different cardinalities, but include many common (i.e., identical here) relational descriptors, and the rank of each relational descriptor is known in each relational repository. ;

[0054] Of course, all the advantageous characteristics described (above or below) with reference to the relational reference system used to analyze the candidate image also apply to the compatible relational reference system(s) used to analyze the reference images.

[0055] It is also possible to consider that a different relational reference frame is used for each distinct image implemented in the process of the invention, whether it is a reference or candidate image, but on the condition that all the relational reference frames implemented are compatible with each other, that is to say that they have at least a common denominator allowing the use of the process.

[0056] For the sake of clarity in the description, the inventors have chosen to maintain the simplest general implementation case, namely, that in which the same relational reference system is used for all images. However, the reader is advised to bear in mind the possibility of implementing different relational reference systems as described above. Thus, according to an advantageous feature of the method according to the invention, it is conceivable that each reference list implemented in step b) could have been pre-established from the same single relational reference system, compatible with the relational reference system implemented in step a) (used to establish the candidate list), but possibly different from the latter.On the contrary, according to another advantageous feature of the method according to the invention, it is conceivable that the relational reference system implemented in step a) (used to establish the candidate list) is identical to the relational reference system used to establish each reference list (this is the case described below).

[0057] It should be noted that the relational reference frame can, in addition to the ordered list of relational descriptors, and, for each relational descriptor, in addition to the calculation method for the chosen transient descriptors and their degree of similarity measure, include supplementary data such as a maximum number (for example, 1, 2, or 3...) of points of interest where each relational descriptor can be found in the candidate image, and possibly in each reference image. More precisely, as explained, the ranking of the descriptors resulting from the calculation method associated with a given relational descriptor allows us to retain the coordinates of the m points associated with the transient descriptor(s) whose degrees of similarity with the relational descriptor in question are the highest, according to the said degree of similarity measure; these m points thus become, in effect, m "points of interest".For any relational descriptor in the ordered list of the relational reference frame, m points of interest can thus be detected in the image as defined by the invention, m generally being an integer greater than or equal to one. When the integer m is strictly greater than 1, it is possible for it to be one of the parameters of the relational reference frame associated with the relational descriptor of said relational reference frame.

[0058] The relational reference frame can also include the coordinates of certain points where we will seek to apply the calculation methods, in the candidate image and possibly in the reference images, to find which relational descriptor best describes this point.

[0059] The relational repository can also, in addition to the ordered list of relational descriptors, and, for each relational descriptor, in addition to the method of calculating the chosen transient descriptors and their degree of similarity measure, include any other type of additional data without changing the meaning of the invention.

[0060] Furthermore, it is perfectly possible, at a given rank in the ordered list of relational descriptors of the relational repository, to consider not just one but several different calculation methods associated with the relational descriptor at that rank. Thus, the calculation method applied to a candidate image and a reference image, and / or to different reference images, is not necessarily the same for a given relational descriptor. When this is the case, the image(s) to which this calculation method is intended to be applied are indicated in an additional parameter within the relational repository.

[0061] According to another advantageous feature of the method according to the invention, the relational repository may include a classification of relational descriptors, that is, their grouping into categories so as to form subsets of relational descriptors describing at least one common characteristic. For example, the relational repository may group together relational descriptors that respectively describe contour, color, or texture characteristics in the image.

[0062] In one embodiment, the relational repository is obtained by extracting information from one or more "repository images", which can be synthetic or natural images.

[0063] For an image to be selected as a reference image, it must possess specific informational characteristics. To date, researchers have identified that an image containing rich information (a variety of local and multiscale features), coupled with regionalized textures or micro-textures, varied contours, and a certain degree of entropy, is a potentially good candidate for becoming a reference image. The ordered list of relational descriptors is then said to originate from a "visually complex" reference image. In a visually complex image, neighboring pixels are uncorrelated, and the distribution of pixels is largely random yet information-rich, allowing different descriptors to characterize each region of the image.An example of a type of natural image that can serve as a reference image is illustrated in the figures accompanying this invention (an iguana in a foliage environment). Conversely, an example of an image that is not ideal, or even suitable, for determining relational descriptors is a Perlin noise image. The reference image is distinct from the candidate image and from each reference image.

[0064] Furthermore, it is conceivable that the relational descriptors included in the ordered list of the relational repository are distinct and different. In practice, this means that they are countable and that no two relational descriptors are identical in the ordered list.

[0065] According to another advantageous feature of the method according to the invention, the relational descriptors included in the ordered list of the relational reference frame are vectors that are equally distributed among themselves in the sense of a defined degree of similarity measure. Such a construction can notably be carried out using a "k nearest neighbors" method.

[0066] According to an advantageous feature of the invention, each reference list is preferably saved for later use. It can also be modified and optimized between two implementations with candidate images.

[0067] According to an advantageous feature of the method according to the invention, the candidate list can be saved for subsequent processing or use. In particular, the processing in step f) may include saving the candidate list in a form usable for computer or automated manipulation, preferably in a form similar to that of the corresponding reference list. Preferably, the reference list includes a record of the coordinates of each point of interest according to a given coordinate system of the reference points. The candidate list then includes a record of the coordinates of the candidate points of interest according to the same coordinate system. Among the coordinate systems, in the context of two- or three-dimensional spaces, examples include the Cartesian coordinate system, the polar coordinate system, and the cylindrical coordinate system.

[0068] Alternatively or in addition, the processing of step f) may include grouping the candidate list and each reference list, all determined from the same ordered list of relational descriptors, into a form usable by computer or automatic processing.

[0069] According to another feature of the method according to the invention, and to facilitate the continuation of the method, the candidate and reference lists are indexed to the ordered list of relational descriptors. That is, these reference and candidate lists include a means of identifying the relational descriptor and / or its position in the ordered list, so as to know which relational descriptor gave rise to the coordinates of the point of interest in question. Such an identification means may, for example, be an indicator of the relational descriptor itself. Alternatively, the identification means may be the number corresponding to the position of the relational descriptor in the ordered list of the relational repository. Alternatively still, the identification means may include ordering the reference and candidate lists in the same order as the ordered list of relational descriptors.

[0070] In step f), the processing of the candidate list may possibly include more elaborate computer or automatic operations, for example matching the candidate list with the reference list, registrating the candidate image with the reference image, classifying the images, calculating relational signatures between the images for unit recognition.

[0071] According to another feature of the invention, the processing in step f) includes a step of determining the existence of corresponding points of interest between the candidate list and each reference list. By " similar points of interest"Points of interest" should be understood as points of interest associated respectively with a descriptor of the reference image and a descriptor of the candidate image, both similar to the same relational descriptor of the relational repository (when the same relational repository is used to analyze each reference image and the candidate image). By "similar," as previously stated, we mean that the degree of similarity between the descriptors is high, according to a chosen similarity criterion. Thus, a candidate point of interest is homologous to a reference point of interest if the descriptors of these points are both similar to the same relational descriptor of the corresponding relational repository. In other words, two points are homologous (from one image to the other) if they are described with degrees of similarity of the same order by the same relational descriptor.Thus, two points of interest are homologous when, one originating from the candidate image and the other from the reference image, their transient descriptors possess sufficient degrees of relevance and degrees of similarity of the same order with respect to the associated relational descriptor in the relational frame. Two homologous points of interest do not, of course, necessarily have the same coordinates in the candidate and reference images.

[0072] When the relational repositories used to analyze each reference image and the candidate image are different but compatible, we can, for example, consider that two points of interest are homologous when they are positioned at the same rank or at the corresponding ranks in each of the lists.

[0073] If corresponding points of interest exist between the images, it is possible to group the coordinates of these corresponding points of interest into pairs. If the same relational descriptor from the ordered list is associated with several points of interest in at least one of the images, it is possible to generate several pairs of coordinates for this relational descriptor to cover all or some of the possible combinations of corresponding points of interest. It is important to note that in each pair of coordinates, one coordinate corresponds to the point of interest of the relational descriptor in the candidate image, while the other coordinate corresponds to the point of interest of the relational descriptor in the reference image.

[0074] The ordered nature of the list of relational descriptors included in the relational repository is advantageous for quickly identifying relational descriptors that have given rise to homologous points of interest in the reference lists and candidate list.

[0075] Each of the lists, candidate or reference, can be analyzed from a statistical point of view by analyzing the easily manipulable data (the coordinates) that they contain, or from a geometric point of view by analyzing the points of interest.

[0076] Thus, according to yet another feature of the process according to the invention, the processing of step f) includes, in addition to or in place of the processing already described, a statistical analysis of the reference points of interest and the candidate points of interest.

[0077] For example, statistical analysis includes a statistical calculation between coordinates within each candidate or reference list, and / or between candidate and reference lists.

[0078] The process of linking images then involves, for example, establishing a mathematical relationship between the coordinates of a point of interest in the candidate list and the coordinates of the corresponding point of interest in the reference list, as well as a statistical calculation on all the mathematical relationships established for all corresponding points of interest from one list to the other. Another example is that the statistical analysis might also involve comparing the points of interest in one list with the points of interest in the other list to see if certain descriptors in the ordered list do not result in any points of interest in the candidate list but result in at least one point of interest in the reference list, or vice versa.

[0079] More precisely, according to this variant in which the processing of step f) includes a statistical analysis, the points of interest are defined by coordinates with m components, and the statistical analysis is conducted on sets, each formed by the coordinates or groups of coordinates of the same rank of the points of interest. Thus, the coordinates of the list (or group of coordinates), placed at the same rank(s) in the analyzed lists, are treated from a statistical point of view. This statistical processing results in data aggregates in each of the candidate and reference lists. By " aggregates"Data," the result of the statistical analysis, in the form of data packages grouped according to chosen criteria. For example, an aggregate might group points of interest based on their geographic location in the candidate or reference image. Another example of an aggregate could be the grouping of points of interest belonging to the same regional texture of an image. A further example of an aggregate is the set of descriptors that have no points of interest in the candidate and / or reference image.

[0080] Preferably, the sets (or aggregates), each formed by the coordinates or groups of coordinates of the same rank of the candidate points of interest, are ranked according to a similarity criterion with respect to the sets, each formed by the coordinates or groups of coordinates of the same rank of the reference points of interest. This ranking allows for the evaluation of the similarity between the candidate image and the reference image. The similarity between the images encompasses the similarity between lists and the similarity between material subjects represented in the images.

[0081] This embodiment, in which the processing of step f) involves statistical analysis, is particularly advantageous when the candidate image and the reference image are taken from the same viewing angle and at a similar magnification, for example, using a target. The images are then said to be pseudo-registered or pre-registered, and their correlation can be efficiently performed by statistical analysis. A more precise example of this embodiment involves calculating the distance between the point of interest associated with a relational descriptor in the reference image and the point of interest associated with the same relational descriptor in the candidate image; that is, calculating the distance between the coordinates of two corresponding points of interest.

[0082] According to another advantageous feature of the method according to the invention, the processing in step f) comprises, in addition to or instead of the processing already described, a geometric analysis including the mapping of candidate interest points from the candidate list with the corresponding reference interest points from each reference list. In cases where, for the same relational descriptor, several coordinates of interest points are associated in a candidate or reference list, it may be advantageous to determine the best mapping from among all possible combinations.

[0083] According to this variant of the processing of step f), the matching is followed by the determination of at least one geometric transformation associating the coordinates defining the points of interest of the candidate list with the coordinates defining the corresponding points of interest of each reference list.

[0084] According to this variant, the matching process involves determining at least one geometric transformation that associates the coordinates of the points of interest in the candidate list with the coordinates of the corresponding points of interest in one of the reference lists. This operation can be repeated with each pair of corresponding points of interest between the candidate list and one of the reference lists, thus determining several geometric transformations to try to align the candidate image with this reference image, and comparing them to determine the best one. This operation can also be repeated with each of the reference lists, thus determining several geometric transformations to try to align the candidate image with each of the reference images, and comparing them to determine the best one.

[0085] The ranking of these geometric transformations can be done according to one or more predetermined quality criteria to choose the one that is best for relating the candidate image and this reference image.

[0086] The quality criteria for determining whether a geometric transformation is better than others include, for example: the least-squares error between the transformed and target images (e.g., the candidate and reference images, respectively), the number of corresponding points of interest, and the ability to map at least a given number of points of interest from the candidate image into a defined area (as small as possible) around the corresponding points of interest in the reference image. For example, one might search for a point within a small area, but allow a certain tolerance radius around the theoretical corresponding point of interest (in the reference image) that one is trying to match with the corresponding point of interest (in the candidate image).In practice, for example, we can consider that the greater the number of corresponding homologous points of interest, each within the smallest possible tolerance radius around their respective homologous point of interest, the higher the quality criterion.

[0087] The desired geometric transformation can, for example, take the form of a rigid transformation (translation and / or rotation and / or scaling), a homography, or any other point-to-point geometric transformation (rigid or non-rigid).

[0088] It is also conceivable that no geometric transformation exists between the candidate image and the reference image, in which case these two images will be considered too far apart to be able to be recalibrated or compared, so that their relationship has only the interest of being able to affirm that they have no link between them.

[0089] The embodiment in which the processing in step f) includes geometric analysis is not incompatible with the preceding embodiment in which the processing includes statistical analysis. Geometric analysis is preferable when the reference image and the candidate image are not pseudo-registered, that is, when they were taken without any precautions, meaning that the candidate image was taken without attempting to reproduce the shooting of the reference image(s).

[0090] Of course, there are as many geometric transformations to determine as there are reference images with which we want to relate the candidate image. These geometric transformations can also be classified according to a pre-established quality criterion to choose the best one.

[0091] In other words, one of the reference images provides access to the most promising geometric transformation for relating the candidate image to this reference image, for example for its future registration or the future recognition of a material subject present in the candidate image.

[0092] According to an advantageous feature of the method according to the invention, each geometric transformation sought in step f) is a direct geometric transformation between the candidate list and each reference list.

[0093] Thus, we seek the geometric transformation that allows us to place the coordinates of one of the points of interest in the candidate list on, or as close as possible to, the coordinates of the corresponding point of interest in each reference list, and this for the maximum number of corresponding points of interest. In other words, the geometric transformation must ensure the best possible match between the coordinates within each pair of corresponding points of interest.

[0094] According to another possible feature of the method according to the invention, each desired geometric transformation is an indirect geometric transformation between the candidate list and the reference list. Such an indirect geometric transformation results from a succession of geometric transformations between the coordinates of the candidate list and the coordinates of the reference list, via at least one intermediate list containing the coordinates, in any intermediate image, different from the candidate image and each reference image, of positions of descriptors of this intermediate image determined according to the calculation method of the relational reference frame and similar to the relational descriptors.

[0095] In simplified terms, an indirect geometric transformation involves: a first direct geometric transformation between the candidate list and the intermediate list, which intermediate list includes the coordinates, in the intermediate image, of the possible points of interest associated with the relational descriptors of the ordered list of the relational reference frame, and, a second direct geometric transformation between the reference list and the intermediate list.

[0096] According to an advantageous feature of the method according to the invention, the best geometric transformation is applied to the candidate image in order to register the candidate image with respect to the reference image. In other words, applying the best geometric transformation allows this candidate image to be registered with respect to the reference image associated with this best geometric transformation.

[0097] Registration is a technique based on matching images or parts of images, or information extracted from these images, allowing comparison, superimposition or even combination of the respective information contained in these images, parts of images or information extracted from these images.

[0098] According to another advantageous feature of the method according to the invention, a step is planned to determine whether the candidate image belongs to a predetermined class. This step is generally subsequent to the processing in which a geometric transformation is determined, enabling the candidate image to be converted into the reference image, or alternatively, is generally subsequent to a statistical analysis step.

[0099] The membership class is represented respectively by the reference list, the reference image, or the material subject depicted in the reference image, whichever is closest according to a chosen membership criterion. Thus, a membership class is defined by various characteristics of the class. It is the similarity to each, or all, of these characteristics, referred to here as the "membership criterion," that determines membership in the class. Therefore, verifying the membership criterion for a class might, for example, consist of evaluating the distance, with respect to the chosen characteristic, between the candidate list and each reference list. For example, the membership criterion might be considered met for the shortest distance(s) from the candidate list to the reference list. This distance between lists can be used in conjunction with a defined acceptance threshold.This amounts to comparing the assessed distance with a defined acceptance threshold. Above the acceptance threshold, the candidate list does not belong to the membership class corresponding to the reference list; below it, it belongs to that class.

[0100] It is entirely possible that the candidate list (or the candidate image or the material subject represented in the candidate image) does not belong to any predefined class.

[0101] A "class" is defined as a set of properties characteristic of a reference list, a reference image, or a physical object represented in the reference image. A class can contain one or more elements (for example, the class of a specific product identified by its unique serial number (a single isolated element), the class of serial products of the same model (numerous constituent elements), the class of products of the same brand, the class of components of an assembled product, etc.). A class with only one member can be used for authentication. This is then called a unitary class. A class can be named and / or associated with a person or organization (for example, the class of physical objects belonging to a person or organization on a specific date), or even with another class. A class can be reconfigurable and depend on the choice of relational repository.The characteristic properties of a class can, of course, be more or less specific. A class can itself be divided into subclasses, disjoint or not, contained within the class in such a way that each subclass includes only certain elements of the class. Thus, each class can be a more or less extensive set, and include different subclasses. The class can be reconfigurable over time, in which case the reconfigured class is considered a new class, although it may have identical characteristics, including the same name.

[0102] According to an advantageous feature of the method according to the invention, it is conceivable that at least some of the reference images are constructed images representing at least two distinct material subjects belonging to the same class of material subjects.

[0103] A constructed image is obtained, for example, by superimposing, after registration for instance, two images representing two material subjects of the same class, and by eliminating the distinct parts to retain only the parts common to these two images. Using a constructed image representing several distinct subjects belonging to the same class makes it possible to eliminate the specificities of each subject, retaining only the features common to all subjects. In other words, the constructed reference image used smooths out the noise present in each image of a single subject. According to an advantageous feature of the method according to the invention, it is also conceivable that at least some of the reference images are constructed images representing at least two distinct material subjects belonging to distinct classes of material subjects.

[0104] Thus, when each reference image is respectively representative of a class of material subjects, it is conceivable to use the processing of the candidate list or the candidate image to qualify the membership of a candidate subject represented in the candidate image to the class of material subjects which is represented by the corresponding reference image.

[0105] In practice, a score can be established for each geometric transformation that converts the candidate image into one of the reference images, based on pre-established criteria. The resulting score provides an initial indication of the similarity between the reference image and the candidate image. Similarity between images encompasses the similarity between lists and / or the similarity between material subjects represented in the images.

[0106] According to another advantageous feature of the method according to the invention, a unitary candidate image recognition step can be implemented.

[0107] Preferably, this unit recognition step is subsequent to the registration step. It is particularly conceivable that the unit recognition step following registration could include a process for determining a relational fingerprint between the two candidate and reference images, as described in document WO2017198950.

[0108] Alternatively, it is also conceivable that the unit recognition step following registration includes a point subtraction step between the registered candidate image and each reference image.

[0109] Alternatively, and according to an advantageous feature of the method according to the invention, the unit recognition step of the candidate image includes iterating steps a) to f) with other reference images and / or another relational repository comprising another ordered list of relational descriptors and / or other methods of calculation and / or other methods of determining the degree of similarity.

[0110] Thus, after establishing an initial class link between the candidate image and the reference image, it is possible to specify which subclass the material subject represented in the candidate image belongs to by repeating steps a) and f) of the process, according to the various advantageous characteristics described above. This iteration is performed using other reference images and / or another relational framework. This iteration allows for refining the recognition of the material subject represented in the candidate image, ultimately leading to a unified recognition of the material subject.

[0111] According to an advantageous feature of the method according to the invention, it is conceivable that each iteration of steps a) to f) is carried out on an area of ​​interest of the candidate image which has a reduced surface area compared to the total surface area of ​​the part of the candidate image initially linked.

[0112] This feature allows us to focus on certain potential details of the material subject represented in the candidate image, once an initial registration has been implemented.

[0113] The invention also relates to a use of the described linking process to recognize different elements on a route in order to establish a digital collection of digital content, said reference images being chosen according to the elements to be recognized on the route.

[0114] More precisely, according to this application, the class to which different elements belong (single or multiple) is successively identified: material subjects or parts of the same material subject. The different elements recognized could, for example, be each side of a packaging box, or each component of a watch. In this application, the reference images are chosen according to the desired membership classes. This application allows the user's path to be followed (in terms of time and / or surrounding space). The path can be either predetermined, meaning that a given element must be recognized before another, or free, meaning that the different elements can be recognized in any order, along the entire path or only a portion of it, or even that at least one element can be recognized multiple times.

[0115] According to this usage, each recognized element is associated with a chosen digital content, either directly related to the object or unrelated to it. For example, the digital content may include the owner of the element, and / or a message intended for the user who recognized the element, and / or a cryptographic token that can be used in computer processes such as blockchain processes.

[0116] Thus, the various elements recognized along the path lead to a collection that can be associated either with the pre-established digital content described above, or with the digital collection of the elements themselves. The data comprising the collection include, for example: the physical object or the part of the physical object recognized, the class to which this object belongs, the digital content associated with this object, the cryptographic token associated with this object, etc.

[0117] Depending on the preferred use case, the successive recognition of each element can trigger access to the digital content (static or dynamic) associated with that element, and all the digital content encountered along a path can be compiled into a digital content collection. This digital content collection can advantageously be managed using cryptographic tokens stored on a blockchain, known as non-fungible tokens (NFTs). This particular implementation allows for the reliable and secure management of ownership of collections and, consequently, of the constituent elements of these collections. This implementation also allows for the automatic and secure tracking of ownership transfers.

[0118] According to an advantageous feature of this use, prior to the recognition of the elements, the digital content associated with each element is to be recorded in memory.

[0119] According to another advantageous feature of this use, it is planned, after the recognition of the elements, to record in a memory or a register (such as a blockchain), the access and / or transfer of ownership of the element, and / or the creation of a cryptographic token, and / or access to a cryptographic token associated with the element.

[0120] Another advantageous feature of this use is that a step is planned to recognize the user and / or the device used to generate the candidate image, prior to the recognition of the elements.

[0121] The invention also relates to a method for developing a reference list comprising the following steps:the implementation of a relational reference system comprising at least: an ordered list of relational descriptors, at least one calculation method to be applied to the reference image to determine descriptors of that image, and a method for determining the degree of similarity between two descriptors, and the determination of descriptors of the reference image, calculated according to each descriptor calculation method of the relational reference system, and the position of each of these descriptors in the reference image, the determination of the degree of similarity, determined according to the method of determining the relational reference system, between each descriptor of the reference image and each relational descriptor of the relational reference system, the determination of a reference list comprising the positions, called reference points of interest, in the reference image,of each descriptor of the reference image exhibiting the greatest similarity to the corresponding relational descriptor, and which is ordered according to the order of the relational reference frame.

[0122] This process makes it possible to establish at least one reference list, preferably a bank of reference lists, prior to the implementation of the matching process according to the invention, which generates a considerable time saving when implementing the matching process with a candidate image.

[0123] This method is very similar to steps c) to e) of the linking method according to the invention, except that a reference image is used instead of the candidate image, and the relational frame used is not necessarily identical to the relational frame used to establish the candidate image, provided that it is compatible with the one used to establish the candidate list. Preferably, however, the relational frames are identical.

[0124] According to an advantageous feature of this development process, a set of reference lists is developed by iterating the determination steps, from the implementation of relational repositories compatible with each other, preferably from the same and unique relational repository.

[0125] This process can advantageously be implemented in distributed computing mode via a cloud environment for calculations and comparisons with reference lists, which can also be advantageously stored in the same cloud. This process can advantageously use smartphones to connect to this environment and transmit candidate images or the results of calculations performed on these candidate images. Similarly, this process can extensively utilize blockchains to enhance the security of the digital data involved.

[0126] Of course, the different characteristics, variants and embodiments of each process according to the invention can be combined with each other in various ways insofar as they are not incompatible or mutually exclusive.

[0127] Furthermore, various other features of the invention become apparent from the attached description made with reference to the drawings which illustrate non-limiting embodiments of the invention and where: [ Fig. 1 ] is a schematic representation of a relational reference system used in the linking process according to the invention, [ Fig. 2 ] is a schematic representation of certain steps in the connection process according to the invention, [ Fig. 3 ] is a schematic representation of an example of linking (step f) of the linking process according to the invention, [ Fig. 4 ] is a schematic representation of another example of linking (step f) of the linking process according to the invention, and [ Fig. 5 ] is a visual representation of the connection method and the connection method according to the invention.

[0128] It should be noted that in these figures the structural and / or functional elements common to the different variants or examples have the same references.

[0129] On the figure 2 , we have represented the main steps of a process for relating at least part of an Ican candidate image with at least one reference image, in accordance with the invention.

[0130] According to this process, the following steps are planned: a) implementation of a relational repository comprising at least: an ordered list of relational descriptors, at least one calculation method to be applied to the image to determine descriptors of that image, and a method for determining the degree of similarity between two descriptors, b) implementation, for each reference image, of a reference list which includes the positions, called reference points of interest, in the reference image, of descriptors of the reference image similar to relational descriptors from a relational repository compatible with the relational repository implemented in step a), which reference list is ordered according to the order of this compatible relational repository, c) determination, in the candidate image, of descriptors of the candidate image calculated according to each descriptor calculation method of the relational repository implemented in step a),and the position of each of these descriptors in the candidate image, d) determination of the degree of similarity, determined according to the method of determining the relational reference system implemented in step a), between each descriptor of the candidate image and each relational descriptor of this relational reference system, e) determination of a candidate list which includes the positions, called candidate points of interest, in the candidate image, of the descriptors of the candidate image exhibiting the greatest similarity with the relational descriptors of the relational reference system implemented in step a), which candidate list is ordered according to the order of this relational reference system, and f) processing of the candidate list with respect to each reference list on the basis of the order of the candidate and reference lists.

[0131] This matching process is carried out using a computer medium such as a processor.

[0132] As shown by figure 2 , it is therefore first planned to implement a pre-established relational reference R (step a).

[0133] This relational repository R is illustrated in the figure 1 It includes an ordered list of relational descriptors Desc 1, Desc 2, Desc 3, ..., Desc N, as well as a calculation method and a method for determining the degree of similarity associated with each of these relational descriptors. The order is given by the references 1, 2, 3, ..., N and represented by the dotted lines on the figure 1 .

[0134] The calculation method associated with a relational descriptor is intended to be applied to an image and results in a local descriptor of that image that describes relevant predefined features of the image located at one or more salient points. This local descriptor is called the image's "transient descriptor."

[0135] The method for determining the degree of similarity between two descriptors allows us to compare the similarity between the relational descriptor and the transient descriptor resulting from the calculation method associated with that relational descriptor. The degree of similarity is higher the more similar the compared descriptors are. For example, one possible method for determining the degree of similarity is to measure a distance, in the sense of Hamming, Mahalanobis, Levenshtein, or Hausdorff, according to the chosen similarity criterion. In other words, the degree of similarity is the value of the measured distance, while the similarity criterion is the nature of the chosen similarity calculation (for example, calculating a distance in the sense of Hamming, Mahalanobis, Levenshtein, or Hausdorff).

[0136] Preferably, the relational and transient descriptors being compared are considered to be similar if the degree of similarity obtained by the method of determination is greater than or equal to a fixed minimum threshold.

[0137] In general, the calculation method associated with a relational descriptor is considered to be either identical or different from one relational descriptor to another, and the method for determining the degree of similarity between two descriptors is also considered to be either identical or different from one relational descriptor to another. Thus, in the relational framework R, there can be at most as many calculation methods and methods for determining the degree of similarity as there are relational descriptors.

[0138] In a specific case not shown, the same calculation method is used for all relational descriptors, and the same method is used to determine the degree of similarity. In this specific case, there is a single calculation method and a single method for determining the degree of similarity.

[0139] As shown by figure 2 The next step is to use the relational repository R illustrated in the figure 1 in order to determine a list, called candidate list Lc, listing the positions, here in the form of coordinates, of points of interest where the relational descriptors are found, in a candidate image Ican that we wish to analyze (steps c to e).

[0140] An ICAN candidate image is, for example, a photograph taken using a mobile phone camera. This image is, for example, stored in memory.

[0141] More specifically, in step c), the descriptors of the candidate image are determined. To do this, the calculation methods of the relational reference frame are applied to the candidate image in order to determine the descriptors of the candidate image, called at this stage "transient descriptors". In this same step, the position of the transient descriptors in the candidate image is also determined; these positions are obtained, for example, by the same calculation used to determine the transient descriptors.

[0142] In step d), the degree of similarity between each transient descriptor found and the relational descriptor associated with the calculation method used is then estimated (when each relational descriptor is associated with its own relational descriptor), or between each transient descriptor found and each relational descriptor of the relational repository (when a single calculation method is used for all relational descriptors). Preferably, at this step, the transient descriptors are ranked according to their degree of similarity with each relational descriptor taken individually, in order to find the most similar transient descriptor(s) to retain for each relational descriptor.

[0143] When two transient and relational descriptors are considered similar, the salient point of the analyzed image where the transient descriptor is located is retained as a point of interest for that relational descriptor. The relational descriptor is then considered to "be present" in the analyzed image at the point of interest. As shown in the figure 2It is entirely possible to find at least one of the relational descriptors at several distinct points of interest in the Ican candidate image. For example, the first relational descriptor Desc1 of the relational reference frame R is found at three distinct points of interest in the Ican candidate image: (x1, y1)C, (x'1, y'1)C and (x"1, y"1)C; the second relational descriptor Desc2 is found at two distinct points of interest in the Ican candidate image: (x2, y2)C and (x'2, y'2)C; the third relational descriptor Desc3 is found at only one point of interest in the Ican candidate image: (x3, y3)C; and the nth relational descriptor DescN is found in two distinct points of interest of the candidate image Ican: (x N , YN ) C and (x' N , y' N ) C .

[0144] In step e), the position of each point of interest is incorporated into the candidate list Lc, that is, the coordinates of each point of interest. This incorporation is based on the order of the relational descriptors given in the ordered list of the relational repository R; in other words, the relationship between the order of the candidate list Lc and the order of the relational descriptor list is known. Specifically, here, the candidate list is ordered according to the order of the relational descriptor list, that is, by respecting the order of the relational descriptors given in the ordered list of the relational repository R. This order is symbolized by the dashed lines in both the candidate list and the ordered list.

[0145] As shown by figure 2 , it is then planned to link the candidate list Lc obtained for the Ican candidate image with at least one reference list L1, L2, L3.

[0146] Preferably, each reference list L1, L2, L3 is obtained from a reference image Iref1, Iref2, ... Irefk, according to the same principle as that described above for obtaining the candidate list Lc, except that the relational repository used can be different from that used to obtain the candidate list Lc, provided it remains compatible with the relational repository used to establish the candidate list Lc. Here, for the sake of simplicity, we assume that the relational repository used to establish the reference lists L1, L2, L3 and the candidate list Lc are identical.

[0147] The reference lists L1, L2, L3 are preferably obtained prior to the implementation of the matching process according to the invention. Thus, it is sufficient to call (or implement) the pre-established reference lists L1, L2, L3 in the matching process according to the invention (step b).

[0148] More specifically, to establish each reference list L1, L2, L3, a process for developing a reference list from a reference image Ref1, Iref2, ... Irefk, according to the invention, is implemented.

[0149] According to this manufacturing process, the following steps are carried out: The implementation of a relational repository comprising at least: an ordered list of relational descriptors Desc1', Desc2', Desc3', ..., DescN', at least one calculation method to be applied to the reference image to determine descriptors of that image, and a method for determining the degree of similarity between two descriptors, and the determination of descriptors of the reference image, calculated according to each descriptor calculation method of the relational repository, and the position of each of these descriptors in the reference image, the determination of the degree of similarity, determined according to the method of determining the relational repository, between each descriptor of the reference image and each relational descriptor Desc1', Desc2', Desc3', ..., DescN' of the relational repository, the determination of a reference list comprising the positions, called reference points of interest, in the reference image, of each descriptor of the reference image exhibiting the greatest similarity with the corresponding relational descriptor Desc1', Desc2', Desc3', ..., DescN' and which is ordered according to the order of the relational repository. .

[0150] In this development process, the relational reference R used is the same as that used in the relationship process, so that the list of descriptors Desc1', Desc2', Desc3', ..., DescN' is identical here to the list of descriptors Desc1, Desc2, Desc3, ... DescN.

[0151] The reference list is considered to be ordered according to the order of the relational descriptors in the relational repository, provided that the relationship between the order of the reference list and the order of the list of relational descriptors is known. A specific case implemented here is to assume that the order of the reference list is identical to the order of the relational descriptors; that is, the reference list is ordered according to the order of the list of relational descriptors.

[0152] The reference images used are, for example (and preferably), obtained from a process similar to that used to obtain the Ican candidate image, namely here from a camera.

[0153] Once the reference lists L1, L2, ... Lk have been obtained, they are therefore implemented in step b) of the linking process according to the invention.

[0154] THE figures 3 And 4illustrate two different examples of relating the candidate lists Lc and the reference lists L1, L2, L3, namely a relationship by statistical calculation and another relationship by determining a geometric transformation.

[0155] For both types of relationship-building, it is best to begin by identifying the corresponding points of interest between the candidate list and each of the reference lists. In practice, these corresponding points of interest are easily identifiable in the lists, since they are ordered according to the order of the relational descriptor list in the relational repository. The corresponding points of interest are therefore represented by the coordinates located at the same position in the respective lists. Thus, all coordinates at position 1, that is, those containing the relational descriptor Desc1, are the coordinates of corresponding points of interest within the meaning of the invention, and the same applies to the coordinates at positions 2, 3, and subsequent N.Identifying homologous points of interest thus amounts in a way to matching the dotted frames located at the same rank in the candidate list and in the reference list, it being understood that when no point of interest has been associated in the Ican candidate image (respectively in at least one of the reference images Iref1, Iref2, ..., Irefk) with one of the relational descriptors of the ordered list, the candidate list (respectively at least one of the reference lists) has a rank left empty.

[0156] Statistical calculation (illustrated in the figure 3 ) allows us to analyze and compare the coordinates of the corresponding points recorded in the candidate list to those recorded in the reference list, rank by rank.

[0157] The geometric transformation (illustrated in the figure 4This process attempts to place each point of interest in the candidate list onto a corresponding point of interest in the reference list. The geometric transformation can, for example, take the form of a homography, but this is not the only possible geometric transformation. The geometric transformation can also include translation, rotation, and / or scaling.

[0158] There figure 5 illustrates the methods of linking and developing reference lists in accordance with the invention.

[0159] In this illustration, the relational reference frame R is obtained from a complex reference frame image, from which the relational descriptors Desc1, Desc2, Desc3, ..., DescN are extracted. More specifically, a chosen computation mode, here the Accelerated KAZE (or A-Kaze) mode, is applied to the reference frame image (arrow F1 of the figure 5), and results in relational descriptors that aptly describe the reference image. The reference image here represents an iguana in a foliage environment. This reference image is visually complex because neighboring tile pixels are uncorrelated with each other, and the tile distribution is largely random yet information-rich, so different relational descriptors qualify each region of this image.

[0160] The calculation method included in the relational repository is the same as the calculation method used to determine the relational descriptors from the repository image.

[0161] According to an unrepresented variant, it is entirely possible to construct the relational descriptors of the relational repository from scratch, without extracting said relational descriptors from a repository image.

[0162] Once the relational repository R is established, the process of developing the reference lists L1, L2, ..., L5 is implemented (arrow F2 on the figure 5 In this example, the development process is implemented using the five reference images Iref1, Iref2, Iref3, Iref4, and Iref5, to which the relational reference frame calculation method is applied (arrow F2). Each reference image represents a class of material subjects. More specifically, the first reference image Iref1 represents a first model of watch, the second reference image Iref2 represents the class of disposable cups, the third reference image Iref3 represents a second model of watch, the fourth reference image Iref4 a third model of watch, and the fifth reference image Iref5 represents a class of postage stamps.

[0163] By applying the calculation method, the coordinates of the points of interest are determined where each relational descriptor Desc1, Desc2, Desc3, ... DescN is found in each reference image Iref1, Iref2, Iref3, Iref4, Iref5. The points of interest are represented by blue dots in the processed reference images.

[0164] The coordinates of the points of interest in the respective reference images Iref1, Iref2, Iref3, Iref4, Iref5 are then stored in respective reference lists L1, L2, ..., L5. In the reference lists, the order (1, 2, 3, ..., N) of the ordered list of relational descriptors Desc1, Desc2, Desc3, DescN of the relational repository R is respected.

[0165] The linking process is then implemented (arrow F4). Specifically, an Ican candidate image is captured or retrieved. The calculation method of the relational reference frame R (arrow F5) is then applied to this Ican candidate image to extract the points of interest where the relational descriptors of the relational reference frame are located. This is equivalent to performing steps c) to e) of the linking process detailed previously.

[0166] The coordinates of the points of interest in the candidate image Ican are then stored in a candidate list Lc. In the candidate list Lc, the order (1, 2, 3, ..., N) of the ordered list of relational descriptors Desc1, Desc2, Desc3, DescN of the relational reference frame R is maintained. The candidate list Lc is stored in a format usable for computer or automated manipulation, preferably in a format similar to that of the reference lists L1, L2, ..., L5.

[0167] Finally, the candidate list Lc is linked to each reference list L1, L2, ..., L5 (arrow F6). Here, the linking consists of matching the points of interest in the candidate list Lc with the corresponding points of interest in each reference list L1, L2, ..., L5, and then determining the best geometric transformation to make the corresponding points of interest match.

[0168] Here, the best geometric transformation is a rotation and scaling of the Ican candidate image. Determining this best geometric transformation then allows us to determine which of the Ican candidate image the candidate belongs to, according to the reference images Iref1, Iref2, Iref3, Iref4, and Iref5. In this case, the candidate image belongs to the class represented by the third reference image, Iref3. Applying the geometric transformation allows us to align the Ican candidate image with the reference image Iref3.

[0169] It is entirely possible then to iterate the processes of relating and developing according to the invention, with other reference images and / or another relational reference frame.

[0170] Thus, after establishing an initial class link between the candidate image Ican and the third reference image Iref3, it is possible to specify which subclass the material subject represented in the candidate image Ican belongs to. This is achieved by reiterating, firstly, the process of generating reference lists to create new lists from new images belonging to the class represented by the third image, and representing subclasses of that class, and secondly, the process of establishing a relationship according to the invention. This iteration is performed here using the same relational frame R, but it would be entirely possible to change it as well. This iteration allows for a more precise recognition of the material subject represented in the candidate image. By further iterating the processes according to the invention, a unitary recognition of the material subject is thus achieved.

[0171] This is not represented, but it is entirely conceivable that the iteration is implemented on an area of ​​interest of the Ican candidate image which has a reduced surface area compared to the total surface area of ​​the part of the candidate image initially linked.

[0172] This is not shown either, but it is advantageous to use the relationship method according to the invention to recognize different material subjects on a pre-established path in order to establish a digital collection of objects, said reference images being chosen according to said pre-established path.

Claims

1. Method for the correlation of at least part of a candidate image (Ican) with at least one reference image (Iref1, Iref2, Iref3, Iref4, Iref5,..., Irefk), comprising the following steps: a) implementing a relational database (R) comprising at least: an organized list of relational descriptors (Desc1, Desc2, Desc3,..., DescN), at least one calculation method to be applied to the candidate image in order to determine the descriptors of this candidate image, and a method for determining the degree of similarity between two descriptors, b) implementing, for each reference image (Iref1, Iref2, Iref3, Iref4, Iref5,..., Irefk), a list of reference (L1, L2,..., L5,..., Lk) that comprises the positions, said reference points of interest, within the reference image (Iref1, Iref2, Iref3, Iref4, Iref5,..., Irefk), reference image descriptors similar to the relational descriptors (Desc1', Desc2', Desc3',..., DescN') from a relational database compatible with the relational database (R), which list of reference (L1, L2,..., L5,..., Lk) is organized according to the order (1, 2, 3,..., N') of this compatible relational database, c) determining, in the candidate image (Ican), candidate image descriptors calculated according to each calculation method for the descriptor from the relational database (R) implemented in step a), and the position of each of these descriptors in the candidate image (Ican), d) determining the degree of similarity, determined according to the method of determination of the relational database (R) implemented in step a) between each candidate image descriptor (Ican) and each relational descriptor (Desc1, Desc2, Desc3,..., DescN) from the relational database (R) implemented in step a), e) determining a candidate list (Lc) that comprises the positions, said candidate points of interest, in the candidate image (Ican), candidate image descriptors presenting the greatest similarity with the relational descriptors (Desc1, Desc2, Desc3,..., DescN) from the relational database (R) implemented in step a), which candidate list (Lc) is organized according to the order (1, 2, 3,..., N) of this relational database (R), f) processing the candidate list (Lc) regarding each list of reference (L1, L2,..., L5,..., Lk) on the basis of the order of the candidate lists and the list of reference.

2. The method for correlation according to claim 1, whereby each list of reference (L1, L2,..., L5,..., Lk) implemented in step b) was pre-established based on a same and unique relational database, compatible with the relational database (R) implemented in step a).

3. The method for correlation according to claim 2, whereby the relational database (R) implemented in step a) is identical to the relational database used to establish each list of reference (L1, L2,..., L5,..., Lk).

4. The method for correlation according to one of claims 1 to 3, whereby the processing of step f) comprises recording the candidate list (Lc) in a form suitable for computer or automated processing, preferably in a form analogous to that of the corresponding list of reference (L1, L2, ..., L5, ..., Lk).

5. The method for correlation according to one of claims 1 to 4, whereby the processing in step f) comprises a step for the determination of the existence of homologous points of interest between the candidate list and each list of reference.

6. The method for correlation according to one of claims 1 to 5, whereby the processing step f) comprises a statistical analysis of the reference points of interest in each list of reference (L1, L2, ..., L5, ..., Lk) and candidate points of interest.

7. The method for correlation according to claim 6, whereby the points of interest are defined by composing M coordinates and the statistical analysis is carried out on groups, each formed by the coordinates or assembly of coordinates of points of interest of the same ranking for points of interest.

8. The method for correlation according to claim 7, whereby the groups, each formed by the coordinates or assembly of coordinates of the same ranking for candidate points of interest, are ranked according to a criterion of similarity in relation to the groups, each formed by the coordinates or assembly of coordinates of the same ranking for reference points of interest.

9. The method for correlation according to one of claims 5 to 8, whereby the processing step f) comprises a geometric analysis comprising the matching of candidate points of interest from the candidate list (Lc) with the homologous reference points of interest from each list of reference (L1, L2, ..., L5, ..., Lk).

10. The method for correlation according to claim 8, comprising a step of determining whether the candidate image (Ican) belongs to a predetermined rank.

11. The method for correlation according to claim 10, comprising a step of unitary recognition of the candidate image (Ican).

12. Use of the method for correlation according to one of claims 10 and 11 in order to recognize different elements on a route to establish a digital collection of digital content, said reference images being selected according to elements to be recognized on the route.

13. Use according to claim 12, according to which it is intended, prior to the recognition of elements, to save in a database the digital content associated with each element.

14. Use according to one of claims 12 and 13, according to which it is intended, subsequent to the recognition of elements, to save in a database or in a register, the access and / or property transfer of the element, and / or the creation of a cryptographic token, and / or access to a cryptographic token associated with the element.

15. Method for the elaboration of a reference list (L1, L2, ..., L5, ..., Lk) based on a reference image (Iref1, Iref2, Iref3, Iref4, Iref5,..., Irefk) comprising the following steps: - implementing a relational database comprising at least: an organized list of relational descriptors (Desc1', Desc2', Desc3',..., DescN'), at least one calculation method to be applied to the reference image (Iref1, Iref2, Iref3, Iref4, Iref5,..., Irefk) in order to determine the descriptors of this image, and a method for determining the degree of similarity between two descriptors, and - determining descriptors for the reference image, calculated according to each calculation method for the descriptor from the relational database, and the position of each of these descriptors in the reference image, - determining the degree of similarity, determined according to the method of determination of the relational database, between each reference image descriptor and each relational descriptor from the relational database, - determining, from a list of reference comprising the position, said reference points of interest, in the reference image (Iref1, Iref2, Iref3, Iref4, Iref5,..., Irefk), each reference image descriptor presenting the greatest similarity with the corresponding relational descriptor (Desc1', Desc2', Desc3',..., DescN') and that is organized according to the order in the relational database.