Method for predicting the value of a target quantity having a geographical dependency for a candidate object

Abstract

The present invention relates to a method for predicting the value of at least one target quantity having a geographical dependency for a candidate object defined by a first vector of structural characteristics and a second vector of geographical characteristics, the method comprising implementing the steps of: (a) selecting, in a database of reference objects, each of which is also defined by first and second vectors, and associated with a known value of said quantity, at least one set of reference objects comparable to the candidate object, by applying at least one information retrieval model trained on the database of reference objects; (b) estimating the value of the target quantity for the candidate object based on the known values of that quantity for the reference objects comparable to the selected candidate object, by applying a predictive neural network comprising at least one attention block.

Description

[0001] Description

[0002] Title of the invention: Method for predicting the value of a target quantity exhibiting a geographical dependence for a candidate object.

[0003] GENERAL TECHNICAL FIELD

[0004] The present invention relates to the field of spatial analysis and geographic regression. More specifically, it concerns a method for predicting the value of a target quantity exhibiting geographic dependence for a candidate object defined by two vectors of structural and geographic characteristics respectively.

[0005] STATE OF THE ART

[0006] A large number of physical or economic phenomena exhibit a geographical dependence, i.e. are not entirely defined by intrinsic parameters, but also by a spatial parameter.

[0007] The same property, for example, would be assigned a different value depending on its location. Similarly, the same industrial site will have different environmental impacts depending on its geographical position. To give one last example, weather phenomena depend not only on atmospheric physical conditions, but also on the location where they occur.

[0008] Predicting a quantity exhibiting such geographical dependence proves to be very complex, even with a large training set.

[0009] We know of "geographic regression" mechanisms adapted to the field of spatial analysis that take geographical dependence into account, whereas classical regression presupposes that the phenomenon is stationary within the studied space. Integrating the geographical dimension often involves using a spatial interpolation mechanism. This consists of combining known neighboring values ​​to form a local estimate. However, the method for selecting these neighbors, often based on the k-nearest neighbors (k-NN) model, can be simplistic. Furthermore, combining the selected values ​​remains a challenge at high dimensions, requiring consideration of numerous structural and geographical factors. Thus, these methods offer several areas for improvement.

[0010] More recently, it has been proposed to use deep learning models, and in particular neural networks coupled with attention mechanisms.

[0011] The document Viana, D., Barbosa, L.: Attention-based spatial interpolation for house price prediction, for example, proposes a hybrid mechanism called ASI (Attention-based Spatial Interpolation) for estimating the value of a property. In this mechanism, the nearest neighbors are weighted based on their similarity in terms of attributes (the "Euclidean" attention block) and their geographical proximity (the "geo-attention" block). A neural network is then applied to the outputs of the attention blocks to obtain a representation of the property (embedding) on ​​the basis of which the value can be predicted.

[0012] This solution is particularly efficient but proves to be very computationally intensive, with the ASI requiring nearly 20,000 parameters. Beyond the simple question of the necessary resources and computation time, such a model is too complex for its decisions to be explainable and therefore to inspire confidence.

[0013] The present invention improves the situation.

[0014] PRESENTATION OF THE INVENTION

[0015] The present invention relates, in a first aspect, to a method for predicting the value of at least one target quantity exhibiting a geographical dependence for a candidate object defined by a first vector of structural characteristics of said object and a second vector of geographical characteristics of said object, the method being characterized in that it comprises the implementation, by data processing means of a first server, of steps of

[0016] (a) Selection, from a database of reference objects stored on data storage means accessible by said first server, each reference object also being defined by a first vector of structural characteristics of said reference object and a second vector of geographical characteristics of said reference object, and associated with a known value of said magnitude, of at least one set of comparable reference objects to said candidate object, by applying to the first vector and / or the second vector of said candidate object at least one information retrieval model trained on said database of reference objects;

[0017] (b) Estimation of the value of said target quantity for said candidate object as a function of known values ​​of said quantity for comparable reference objects selected, by application of a prediction neural network comprising at least one attention block, step (b) comprising at least one step (b2) of obtaining a weight of each comparable reference object according to a proximity of the first vector of the comparable reference object with the first vector of the candidate object and / or a proximity of the second vector of the comparable reference object with the second vector of the candidate object, by application of said attention block.

[0018] According to advantageous and non-limiting characteristics:

[0019] The said predictive neural network further includes at least one encoding block upstream of the attention block, step (b) comprising a prior step (b1) of obtaining a representation of each candidate object or comparable reference object, by applying said encoding block to the first and / or second vector(s) of the candidate object and comparable reference objects, said attention block being applied in step (b) to said representations of the candidate object and comparable reference objects.

[0020] Said encoding block includes at least one bi-encoder, in particular a multilayer perceptron, step (b1) comprising, for each comparable reference object, obtaining a representation of said comparable object, by applying said encoding block to both the first and / or second vector(s) of the candidate object, and to the first and / or second vector(s) of said comparable reference object.

[0021] The attention block implements a similarity calculation, step (b2) comprising, for each comparable reference object, obtaining the weight of said comparable reference object, by calculating the similarity between the representation of said comparable reference object and the representation of the candidate object.

[0022] Preferably, said similarity calculation is a dot product, i.e. the attention block implements a dot product calculation, step (b2) comprising, for each comparable reference object, obtaining the weight of said comparable reference object, by the dot product of the representation of said comparable reference object and the representation of the candidate object.

[0023] The said prediction neural network comprises a single encoding block and a single attention block, the encoding block being applied to both the first and second vectors of the candidate object and comparable reference objects.

[0024] Step (a) includes the selection:

[0025] - a first set of reference objects comparable to said candidate object in terms of structural characteristics, by applying at least to the first vector of said candidate object a first information retrieval model trained on said database; and

[0026] - a second set of comparable reference objects to said candidate object in terms of geographical characteristics, by applying at least to the second vector of said candidate object a second information retrieval model trained on said database.

[0027] The information retrieval block(s) and the prediction neural network form a Retrieval-Enhanced Machine Learning, REML, type set.

[0028] These structural characteristics of the object are not geographical.

[0029] The said attention block includes a first ancillary block, step (b2) comprising the adjustment, for each comparable reference object, of the weight of said comparable reference object by application of said first ancillary block.

[0030] The said predictive neural network further includes a second ancillary block, step (b2) comprising, for each comparable reference object, the adjustment of the known value of the target quantity for said comparable reference object by application of said second ancillary block.

[0031] The predictive neural network includes an aggregation layer at the output of the attention block, the process comprising a step (b3) of computing a weighted sum of the known values ​​of the target quantity for each comparable reference object, by said weights of each comparable reference object, called the raw aggregation value, by application of said aggregation layer; the value of said target quantity for said candidate object being estimated either directly as said raw aggregation value, or as a function of said raw aggregation value.

[0032] The said prediction neural network further includes at least one decoding block downstream of the attention block, step (b) including a step (b-4) of adjusting said raw aggregation value by applying said decoding block to a so-called global representation of the candidate object.

[0033] The decoding block comprises a multilayer perceptron and a security mechanism using a bounded function, typically a hyperbolic tangent, with step (b-4) comprising the calculation of an adjustment factor to be applied to the raw aggregation value. The method includes a step (aO) of training, by means of data processing equipment on a second server, the information retrieval model and the prediction neural network, from the reference object database.

[0034] According to a second aspect, the invention relates to a first server for predicting the value of at least one target quantity exhibiting a geographical dependence for a candidate object defined by a first vector of structural characteristics of said object and a second vector of geographical characteristics of said object, characterized in that it comprises data processing means configured to:

[0035] - Select, in a database of reference objects stored on data storage means accessible by said first server (1), each reference object also being defined by a first vector of structural characteristics of said reference object and a second vector of geographical characteristics of said reference object, and associated with a known value of said quantity, at least one set of reference objects comparable to said candidate object, by applying to the first vector and / or the second vector of said candidate object at least one information retrieval model trained on said database of reference objects;

[0036] - Estimate the value of said target quantity for said candidate object as a function of the known values ​​of said quantity for the reference objects comparable to said candidate object selected, by application of a prediction neural network comprising at least one attention block, including obtaining a weight of each comparable reference object according to a proximity of the first vector of the comparable reference object with the first vector of the candidate object and / or a proximity of the second vector of the comparable reference object with the second vector of the candidate object.According to a third and a fourth aspect, the invention relates to a computer program product comprising code instructions for executing a process according to the first aspect of predicting the value of at least one target quantity exhibiting a geographical dependence for a candidate object defined by a first vector of structural characteristics of said object and a second vector of geographical characteristics of said object; and a computer-readable storage means on which is stored a computer program product comprising code instructions for executing a process according to the first aspect of predicting the value of at least one target quantity exhibiting a geographical dependence for a candidate object defined by a first vector of structural characteristics of said object and a second vector of geographical characteristics of said object.

[0037] PRESENTATION OF THE FIGURES

[0038] Other features and advantages of the present invention will become apparent upon reading the description of a preferred embodiment which follows. This description will be given with reference to the accompanying drawings in which:

[0039] [Fig. 1] Figure 1 is a diagram of a system for implementing the process according to the invention;

[0040] [Fig.2] Figure 2 is a flowchart illustrating the steps of an embodiment of the process according to the invention;

[0041] [Fig.3] Figure 3 illustrates a predictive artificial neural network structure used in a preferred embodiment of the method according to the invention.

[0042] DETAILED DESCRIPTION

[0043] Architecture The present invention relates in a first aspect to a method for predicting the value of at least one target quantity exhibiting a geographical dependence for a candidate object, with reference to figure 1.

[0044] The process can predict one target quantity, or several together (as long as they relate to the same target object), we will come back to this later.

[0045] We assume that we have a database of reference objects stored on data storage devices 12 accessible by a first server 1, forming a training set. Note that the devices 12 could in practice be those of the first server 1 (which is the execution server of the process as we will see) as represented in Figure 1 (for example, a memory such as a hard drive) or could be distinct / distant from the first server 1 and only connected to it, for example via a communication network 20 such as the internet.

[0046] It is understood that the candidate object and the reference objects are of the same type, the difference being that the reference objects are each associated with a known, for example, measured, value of the target quantity, called the "expected value," whereas this value is generally unknown and therefore needs to be predicted for the candidate object. However, a reference object can certainly be treated as a candidate object, i.e., as if the value of the target quantity for that object were unknown, for example, during the learning phase, which will be described later. In the case of multiple target quantities, it is assumed that a known value of each target quantity is available for each reference object.

[0047] Note that the objects and the target quantity(ies) can be of any nature of interest (the objects are described in detail later), as long as each target quantity has a geographical dependence, i.e. its value has a spatial correlation.

[0048] In this respect, each object (candidate or reference) is defined by a first vector of structural characteristics of said object, of dimension di, and a second vector of geographical characteristics of said object, of dimension d2.

[0049] By structural characteristics of said object, we mean intrinsic characteristics, with "structural" understood in its primary sense (and not as "relating to a physical structure" of the object), and designating intrinsic characteristics. To rephrase, the structural characteristics of the object are non-geographic attributes, as opposed to the geographic characteristics of the second vector. Regardless of the object, these geographic characteristics of the second vector are advantageously coordinates (latitude, longitude), but they could alternatively be, for example, an address, or the distances to predefined points of interest (POIs).

[0050] For example, the object could be a property, and the quantity could target its price, CO2 emissions, energy consumption, etc. The structural characteristics of the first vector would then be its surface area, number of floors, construction materials, etc. Any useful characteristics could be chosen.

[0051] Note that for the reference objects, there can also be a third vector of relative characteristics of that object, of dimension ds, where "relative" means relative to the candidate object. For example, the distance between the candidate object and the reference object could be a relative characteristic. In contrast, all the characteristics of the first and second vectors are absolute characteristics. Naturally, the third vectors depend on the candidate object and cannot be stored in the database; the relative characteristics that compose them are, for example, calculated on the fly (based on the first and second vectors of the two objects). Of course, there is no third vector of relative characteristics for the candidate object itself (that would be meaningless).

[0052] As shown in Figure 1, in addition to the first server 1, a client terminal 10, such as a workstation or smartphone, is advantageously used. This terminal provides an interface through which the user can interact, and a possible second training server 2 may also be used. It is understood that the first server 1 is connected to the client terminal 10 and to the possible second server 2, for example, via a network 20, such as the internet.

[0053] The first / second servers 1, 2 include data processing means 11, 21 such as a processor of any kind, and data storage means 12, 22 i.e. a memory.

[0054] Note that the second server 2 may, if necessary, be confused with the client terminal 10 and / or with the first server 1.

[0055] Process

[0056] With reference to Figure 2, the present process is implemented by the data processing means 11 of the first server, and advantageously begins by obtaining said first and second vectors of the candidate object from the client terminal 10, for example by direct input on an interface of the terminal 10.

[0057] The process then includes a step (a) of selecting, from the database of reference objects, at least one set of comparable reference objects to the candidate object. As will be seen, "comparable" means close in terms of structural characteristics (i.e., "similar") and / or geographical characteristics (i.e., "neighboring"). In the following description, we will refer to comparable objects to designate more precisely those objects comparable to the candidate object.

[0058] This selection process was already known and traditionally implemented using a basic algorithm (typically based on a predetermined metric such as Euclidean distance for proximity in terms of structural features or orthodromic (haversine) distance for geographical and / or predetermined features), with a predefined number of comparable objects, for example, 60 in the document Viana, D., Barbosa, L.: Attention-based spatial interpolation for house price prediction mentioned earlier. Indeed, this high number is necessary for the algorithm's good performance, but it is also the main cause of its complexity, in addition to the large number of parameters in the prediction model.

[0059] The present invention advantageously solves the problem by providing for said selection by applying to the first vector and / or the second vector of said candidate object at least one information retrieval model trained on said database. In practice, the training of said retrieval model is an intelligent indexing of said database.

[0060] An information retrieval model is a model adapted to the task of extracting relevant data from a database, known as "information retrieval". The information retrieval model takes as input the first vector and / or second vector of the candidate object and identifies comparable, i.e., similar, reference objects based on these vectors, and in particular returns the first and / or second vectors of these comparable reference objects.

[0061] An information retrieval model can be seen as a "search engine" returning relevant objects for a candidate object. It is the development of new models of this type that allows for the refinement of an approach that has been, until now, more naive in the literature.

[0062] The use of these models within a Retrieval Enhanced Machine Learning (REML) structure allows, in particular, the development of new selection techniques such as vector search.

[0063] These structures are described in the document Retrieval-Enhanced Machine Learning, Hamed Zamani, Fernando Diaz, Mostafa Dehghani, Donald Metzler, Michael Bendersky.

[0064] While the primary objective of information retrieval models is efficiency in massive databases, here they are used for their relevance and the quality of their selection. In practice, the model can be trained to return a few well-chosen objects, ideally fewer than ten (this number can vary depending on the candidate object). It has been observed that the selection thus achieved yields more relevant references than more naive approaches.

[0065] Preferably, we use a pair of information retrieval models (denoted first information retrieval model and / or second information retrieval model). However, the structure also works with only one of them.

[0066] Step (a) then includes the selection:

[0067] - a first set of reference objects comparable to said candidate object in terms of structural characteristics, by applying at least to the first vector of said candidate object the first information retrieval model trained on said database (more precisely on the first vectors of the reference objects); and / or

[0068] - a second set of reference objects comparable to said candidate object in terms of geographical characteristics, by applying at least to the second vector of said candidate object an information retrieval model trained on said database (more precisely on the second vectors of the reference objects).

[0069] To rephrase, the first information retrieval model can select comparable objects based solely on their structural characteristics, while the second can select comparable objects based solely on their geographical characteristics.

[0070] Alternatively, and preferably, the first information retrieval model selects comparable objects using both structural and geographic characteristics, while the second always selects comparable objects solely on the basis of their geographic characteristics.

[0071] We could very well have just one single information retrieval model selecting comparable objects either on the basis of their structural characteristics alone or solely geographical characteristics alone, or both on the basis of their structural and geographical characteristics, or even a hybrid scheme with on one side an information retrieval model (in particular the first) and a traditional algorithm (for example to select geographical neighbors).

[0072] We can even consider having more than two information retrieval models, for example by searching for comparable objects based on only certain structural characteristics (i.e. by slicing the first vector), or simply with the same characteristics, but a different encoding.

[0073] The implementation with at least two information retrieval models is particularly preferred because it allows for leveraging both types of proximity and achieving the best results. Known algorithms, notably Attention-based spatial interpolation (ASI), select only geographical neighbors and therefore ignore structurally similar but geographically more distant objects. Other methods allow for this, but impose a manually chosen metric, which is difficult to adapt to a new dataset and is not optimal.

[0074] If we have several sets of comparable objects selected, these sets are pooled in the rest of the procedure, either directly, or at the end of step (b2), see below.

[0075] Next, the process includes a step (b) of estimating the value of said target quantity for said candidate object as a function of known values ​​of said quantity for the selected comparable reference objects (one or more sets), by applying a prediction neural network (the prediction model, also called the "downstream" model) comprising at least one attention block, which can be implemented in any known way, although preferred embodiments, particularly suited to REML models, will be seen later.

[0076] The attention mechanism is a popular technique used in deep learning models for many tasks, such as object detection in images, large language models, and recommendation engines. The underlying idea is to help the predictive model focus its attention on the most useful comparable objects when generating its output.

[0077] In the case of a plurality of target quantities, all or part of step (b) can simply be repeated as many times as there are quantities to be estimated.

[0078] Step (b) includes at least one step (b2) of obtaining a weight of each comparable reference object according to a proximity of the first vector of the comparable reference object with the first vector of the candidate object and / or a proximity of the second vector of the comparable reference object with the second vector of the candidate object, by application of said attention block.

[0079] This weight constitutes an attention score, i.e. a scalar, representing a level of proximity of the comparable object with the candidate object, so that the set of weights of the comparable objects forms an attention vector (of size equal to the number of comparables selected).

[0080] According to a first embodiment, this attention block can be applied directly to the first and / or second vector(s) of the candidate object and comparable reference objects. Similar to the ASI mechanism, two attention blocks can even be used: one applied to the first vectors (allowing for a first weight of each comparable reference object based on the proximity of the first vector of the comparable reference object to the first vector of the candidate object), and the other applied to the second vectors (allowing for a second weight of each comparable reference object based on the proximity of the second vector of the comparable reference object to the second vector of the candidate object). This results in two attention vectors, each the size of a group of reference objects. Again, as with information retrieval models, even more could be implemented.

[0081] According to a second, preferred embodiment, the prediction neural network further includes at least one encoding block (or encoder block) upstream of the attention block; that is, a block for generating a "representation" of the input data, called an "embedding," typically in the form of a vector or a matrix feature map. Such encoding blocks are well-known and facilitate the attention step and ultimately the prediction. The encoding block may be pre-trained (and possibly fine-tuned afterward) or trained from scratch.

[0082] Then, step (b) includes a preliminary step (b1) of obtaining a representation of each candidate object or comparable reference object, by applying said encoding block to the first and / or second vector(s) of the candidate object and comparable reference objects.

[0083] The said attention block is then applied in step (b) to said representations of the candidate object and comparable reference objects.

[0084] Like attention blocks, you can have several of them and / or as many as attention blocks, but anything is possible:

[0085] - 1 encoding block (applied to first and second vectors) and 1 attention block as in the example in figure 3;

[0086] - 2 encoding blocks (applied respectively to the first and second vectors) and 2 attention blocks;

[0087] - 2 encoding blocks and 1 attention block applied to a concatenation of the representations obtained by the two blocks for a comparable object

[0088] - etc.

[0089] In a particularly preferred manner, said encoding block includes at least one bi-encoder, in particular a multilayer perceptron (MLP), advantageously implementing a SELU (Scaled Exponential Linear Unit) type activation function.

[0090] A dual encoder is known to increase robustness. It applies vectors from the candidate object and comparable reference objects in the same way, so that they are represented homogeneously.

[0091] In other words, step (b1) comprises, for each comparable reference object, obtaining a representation of said comparable object by applying said encoding block to both the first and / or second vector(s) of the candidate object and the first and / or second vector(s) of said comparable reference object. For example, in the case of a single encoding block, denoting Fi as the absolute features (i.e., first and / or second vector) of the i-th comparable object, Fc as those of the candidate object, and Zi and Z as c the corresponding representations, we have Zi = BiEncoder(Fi) and Z c = BiEncoder(Fc).

[0092] With regard to the attention block, it advantageously implements a dot product calculation, particularly on vector representations.

[0093] Then, step (b1) preferentially includes, for each comparable reference object, obtaining the weight of said comparable reference object by the dot product of the representation of said comparable reference object and the representation of the candidate object.

[0094] Mathematically, we have ai = Zi.Zc, with ai the weight of the i-th comparable object.

[0095] In a conventional way, the weights can be normalized for example by using a softmax function (so that their sum is equal to 1), and undergo other treatments which will be described later.

[0096] It will be understood that in all cases (with or without encoding block) each attention block is applied directly or indirectly to the first and / or second vector(s) of the candidate object and comparable reference objects.

[0097] According to a preferred variant, each attention block can be extended to include features that are not directly usable in the initial encoding, i.e., relative features (of the third vector). Indeed, an absolute vector representation of each object is required for the attention block, and any relative feature cannot be included. However, these features are important; for example, the distance between a candidate object and a reference object.

[0098] We can therefore use a first auxiliary block called a gate to adjust attention a posteriori, this time using all available features, i.e., including those of the third vector. This additional block allows the adjustment of the raw attention weights by a multiplicative factor obtained, for example, via a multiplicative programming model (MLP) using, for each comparable reference object, the concatenation of all the absolute and relative features of the vectors of the candidate object (first and second vectors – there are no relative features for the candidate object), the vectors of the comparable reference object (first, second, and third vectors), as well as the known value of the latter's target quantity.

[0099] This gives us a modified attention weight a'i=ai*annexi(F c( ® Fi ® Ri ® Vi) with Ri the relative features of the i-th comparable object and vi the known value of the target quantity. In the case of a plurality of target quantities, we denote vi,k the known value of the k-th quantity for the i-th comparable object.

[0100] If the doors are used, it is therefore this weight a'i that will be normalized by applying a softmax function rather than ai.

[0101] Alternatively or in addition, a second auxiliary block can be used, which can be comparable to the first auxiliary block (gate), or similar to the decoder block described later. Step (b2) may also include, in parallel, an adjustment of the known values ​​of the target quantity for the reference objects with respect to the candidate object. The idea is to obtain, for the remainder of the process, modified known values, similar to attention weights, by applying this second auxiliary block to the known values ​​of the reference objects. Preferably, this second auxiliary block can also take the form of a Multiplicity Programming (MLP) function, again using, for each comparable reference object, the concatenation of all the absolute and relative characteristics of the vectors of the candidate object, the vectors of the comparable reference object, and the known value of the latter's target quantity.We thus obtain a known modified value v*i = Vi*annex2(F. c ® Fi ® Ri ® Vi).

[0102] Note that in the case of a plurality of target quantities to be estimated, it is understood that the attention weight ai is common (i.e. the step (b1) can be implemented only once even if there are several target quantities), but a modified attention weight and / or a modified known value can possibly be calculated for each target quantity (denoted respectively a'i,k and v*i,k for the k-th target quantity).

[0103] In step (b), a crude estimate of the target quantity is given as a function of the weights of the comparable reference objects, the first and / or second vectors of the comparable reference objects, and the known values ​​of the quantity for these comparable reference objects. Preferably, using the calculated weights (where applicable modified and / or normalized and / or specific to a target quantity – we will retain the notation ai for convenience), a weighted sum (called aggregation – by a dedicated aggregation layer outputting the attention block) of the known values ​​of the target quantity (also where applicable modified – we will also retain the notation vi for convenience) is performed in step (b3) for each comparable reference object v. c = Z en. VL We can apply the same formula v c ,k = Z ai,k. vi,k. for each target quantity.

[0104] This value v c, called the raw aggregation value, already represents a valid estimate of the target quantity. In other words, according to a first embodiment, we stop at the end of step (b3), and said raw aggregation value is returned as the estimate of the value of said target quantity for said candidate object.

[0105] Advantageously, a decoding block can be used to refine this raw value by taking advantage of a global representation also calculated on the basis of attention weights, said decoding block being at the output of the aggregation layer and the attention block.

[0106] Step (b) then includes a step (b-4) of adjusting said raw value of said target quantity by applying said decoding block to the so-called global representation of the candidate object constructed as a function of the weights of said comparable reference objects (the attention vector) and the first and / or second vectors of the comparable reference objects (or directly from their representations).

[0107] This overall representation includes both the specific characteristics of the candidate object, the weighted sum of the characteristics of each reference object, and optionally the raw aggregation i.e. v c previously described. The resulting vector is therefore of size 2di + 2d2 (+ 2ds) and optionally 2di + 2d2 (+ 2ds) + 1.

[0108] If we have several attention blocks, we can obtain a global representation for each attention block and concatenate them. For example, in the embodiment with two attention blocks respectively dedicated to the first vector and the second vector, we obtain a first global representation of the candidate target object (of size m) constructed as a function of the first attention vector and the first vectors of the comparable reference objects, and a second global representation of the candidate object (of size n?) constructed as a function of the second attention vector and the second vectors of the comparable reference objects.

[0109] The decoder typically consists of a novel Multilayer Perceptron type neural network, preferably using a SELU type activation function.

[0110] This decoder allows refining the raw aggregation value v cto adjust it in light of the differences between the candidate object and the reference objects. It allows obtaining a so-called refined value, denoted v c *.

[0111] Preferably, this refinement is performed by applying a multiplicative factor denoted adjc. The decoder predicts the value of this factor through the successive application of the evoked perceptron and a bounded function such as the hyperbolic tangent. This latter function allows the adjustment factor to be contained between fixed bounds such as [-1, 1] (depending on the application), representing either a 100% increase or decrease in value. It therefore acts as a safety mechanism against extreme values ​​that this factor could take. For example, the overall representation is denoted F' C =Z ai (F c ® Fi ® Ri® vi) and adjc=Tanh(Decoder(F' c )). Thus, v c *= Vc.adjc. Alternatively, we can choose another global representation such as F' c= Z ai (Fi ® Ri® vi) ® Fc, the person skilled in the art will not be limited to a particular expression. Again, this work can be carried out target magnitude by target magnitude with typically v c ,k*= v c ,k.Tanh(Decoder(Z Oi,k (Fc ® Fi ® Ri® Vi,k))).

[0112] The user receives in return, typically on terminal 10, the predicted value of the target quantity, i.e. the raw aggregation value(s) v c (v c ,k) or the refined value(s) v c * (v c ,k*) if there has been implementation of step (b4).

[0113] Tests show that the performance (median absolute balanced relative error (MdABRE) between 6 and 10% depending on the dataset) of an ASI with 20,000 parameters is achieved with a predictive neural network of fewer than 800 parameters, or 25 times fewer. It is noted that even better performance could therefore be achieved by increasing the number of parameters, while remaining well below the number of parameters of the ASI.

[0114] By comparison, traditional methods such as regression, kNN or XGBoost can reach 25% MdABRE.

[0115] Explainability

[0116] An additional interesting aspect of the model used by the present process is that it allows us to understand efficiently and reliably which parameters of the objects (i.e. which "features", that is, which structural and / or geographical characteristics) cause them to be chosen in step (a) and how they relate to the final prediction.

[0117] This is important for having confidence in the result. Furthermore, one may realize that certain unexpected parameters prove critical, allowing for a focus on these. Conversely, one may discover that in some cases, particularly in certain geographical situations, a structural characteristic will increase the value of the target variable, and in other cases, decrease it.

[0118] In this respect, the process advantageously includes a step (c) of calculation, for each characteristic of the first vector and the second vector, of an importance score of said characteristic in said estimated value of said target quantity, by application of an explainability model to the first vector and / or the second vector of the candidate object.

[0119] Note that the explainability model (or others) can be applied, either alternatively or as a complement, directly to the intermediate features obtained as output from a block of the predictive neural network, thus providing explainability at the level of each block. Indeed, the proposed model is considered interpretable because it allows access to vectors of intermediate values, which enable a more nuanced understanding of the result.

[0120] The said importance score of a characteristic quantifies the "effect" of the characteristic on the predicted value of the target quantity, and can have a positive or negative value:

[0121] - If it is positive, it means that the characteristic to which it is associated contributes to increasing the value of the target quantity;

[0122] - If it is negative, it means that the characteristic to which it is associated contributes to decreasing the value of the target quantity (in other words, this characteristic goes against the target quantity);

[0123] - If its absolute value is close to zero, it means that the value of the characteristic has no impact

[0124] - On the contrary, if its absolute value is high, it means that the characteristic contributes strongly.

[0125] Many explainability models are known, but preferably this one is a SHAP model, i.e. the said importance score is a so-called SHAP (SHapley Additive exPlanation) value, as described in the document A Unified Approach to Interpreting Model Predictions, Scott M. Lundberg and Su-In Lee, based on the Shapley value in game theory. The SHAP value is additive, so a prediction can be written as the sum of the different importance scores of the features added to a fixed base value.

[0126] The only disadvantage of SHAP is the computational cost, so advantageously we use the Gradient SHAP implementation which allows us to take advantage of the gradients produced by the model to estimate the Shapley values ​​more quickly.

[0127] Furthermore, we cleverly leverage the fact that the current process is based on comparables. Indeed, the SHAP algorithm requires "baselines" against which to compare itself. The general practice uses the entire database of reference objects for this purpose. We can refine this approach by using only comparables, which are more locally relevant.

[0128] Learning

[0129] The process advantageously includes a training step (aO), i.e., learning the parameters of the information retrieval model(s) and the prediction neural network. As explained, the encoding block can be pre-trained, so that it may be a matter of training only the attention block and the decoding block; the encoding block can then be fine-tuned, but preferably all three blocks are trained completely.

[0130] This learning is implemented by the data processing means 21 of the possible second server 2, in a supervised manner, i.e. from the reference objects for which we have a known value of the target quantity, used as ground truth.

[0131] The information retrieval model and the prediction neural network can be trained separately (for example, using the weights obtained from the attention block as an evaluation metric for the retrieval model), but preferably they are trained together (i.e. with a single cost function), possibly varying the learning parameters alternately over time.More precisely, for a plurality of reference objects, mirror steps of steps (a) and (b) are implemented. Advantageously, this involves selecting at least one set of reference objects comparable to a candidate object by applying the information retrieval model to the first and / or second vector of said candidate object; estimating the value of said target quantity for this candidate object using the prediction neural network; and adapting the parameters of the information retrieval model and / or the prediction neural network (or at least the blocks being trained) based on the known value of the associated quantity in the database (feedback loop with respect to the estimated value). In other words, the aim is to minimize a cost (loss) function, which is typically the distance between the predicted value and the known value as the expected value for a reference object.Naturally, if adjustments are used (additional block(s) and / or decoder block), the aim is to minimize the distance between the adjusted values, and where appropriate, an additional cost function can be used.

[0132] Indeed, due to the aggregation process, the individual adjustments made to the value of each comparable are not necessarily in the direction of the final predicted value, and opposing effects can cancel each other out and still yield a good result. However, this reduces usability because it is a less intuitive mechanism. To mitigate this undesirable effect, we propose adding a supplementary cost function (called an intermediate cost function) to the basic cost function (distance). This function forces each element individually to predict the value known after adjustment, thus guiding the adjustments in the "right" direction. More precisely, this supplementary cost function can be the distance between the predicted value and the "modified" known value, i.e., the value obtained by passing through the second annex block (in particular, by applying the formula v* = v*annex2(Fc > F > R > v).

[0133] This additional cost function can be adjusted more or less strongly relative to the initial base cost function to avoid excessively impacting the final performance. The model then behaves somewhat like an expert mixture, with each comparable adjustment representing its own estimate, which is subsequently aggregated.

[0134] The network is considered properly trained when it is able to predict the known value with a sufficiently small margin of error.

[0135] It is noted that, in the embodiment where we wish to estimate a plurality of target quantities, we preferentially train the model simultaneously to predict all these target quantities, so as to have a truly versatile model, in particular capable of choosing "versatile" comparables in view of the various target quantities that can be considered.

[0136] To do this, we can use a slightly more complex cost function that minimizes the distances for each target quantity between the predicted value and the known value as the expected value for a reference object. For example, we can take a total distance per reference object equal to the sum of the distances for each target quantity.

[0137] This is perfectly compatible with any intermediate cost functions.

[0138] Server

[0139] According to a second aspect, the invention relates to the first server 1 for the implementation of the method according to the first aspect.

[0140] Thus, this first server 1 comprises, as explained, at least data processing means 11 and at least access to a memory 12 (storing a database of reference objects in which each reference object is also defined by a first vector of structural characteristics of said reference object and a second vector of geographical characteristics of said reference object, and associated with a known value of said quantity). It is typically a server for predicting values ​​of the target quantity. The data processing means 11 are configured to implement steps consisting of:

[0141] - Select, in the reference object database, at least one set of reference objects comparable to said candidate object, by applying to the first vector and / or the second vector of said candidate object at least one information retrieval model trained on said reference object database;

[0142] - Estimate the value of said target quantity for said candidate object as a function of the known values ​​of said quantity for the reference objects comparable to said candidate object selected, by application of a prediction neural network comprising at least one attention block, including obtaining a weight of each comparable reference object according to a proximity of the first vector of the comparable reference object with the first vector of the candidate object and / or a proximity of the second vector of the comparable reference object with the second vector of the candidate object.

[0143] According to a third aspect, the invention proposes a system comprising said first server 1, as well as the second server 2 and / or a client terminal 10 connected (via network 20).

[0144] The second server 2 includes data processing means 21 configured to implement the learning of the information retrieval model and / or the prediction network.

[0145] computer program product

[0146] According to a sixth and seventh aspect, the invention relates to a computer program product comprising code instructions for the execution (on the data processing means 11 of the first server 1) of a method according to the first aspect of predicting the value of a target quantity exhibiting a geographical dependence for a candidate object defined by a first vector of structural characteristics of said object and a second vector of geographical characteristics of said object; as well as computer-readable storage means (for example the data storage means 12 of the first server 1) on which this computer program product is found.

Claims

27 DEMANDS 1. A method for predicting the value of at least one target quantity exhibiting a geographical dependence for a candidate object defined by a first vector of structural characteristics of said object and a second vector of geographical characteristics of said object, the method being characterized in that it comprises the implementation, by data processing means (11), of a first server (1), of steps of (a) Selection, in a database of reference objects stored on data storage means (12) accessible by said first server (1), each reference object also being defined by a first vector of structural characteristics of said reference object and a second vector of geographical characteristics of said reference object, and associated with a known value of said magnitude, of at least one set of comparable reference objects to said candidate object, by application to the first vector and / or the second vector of said candidate object of at least one information retrieval model trained on said database of reference objects; (b) Estimation of the value of said target quantity for said candidate object as a function of known values ​​of said quantity for comparable reference objects selected, by application of a prediction neural network comprising at least one attention block, step (b) comprising at least one step (b2) of obtaining a weight of each comparable reference object according to a proximity of the first vector of the comparable reference object with the first vector of the candidate object and / or a proximity of the second vector of the comparable reference object with the second vector of the candidate object, by application of said attention block.

2. A method according to claim 1, wherein said predictive neural network further comprises at least one encoding block in upstream of the attention block, step (b) comprising a prior step (b1) of obtaining a representation of each candidate object or comparable reference object, by applying said encoding block to the first and / or second vector(s) of the candidate object and comparable reference objects, said attention block being applied in step (b) to said representations of the candidate object and comparable reference objects.

3. Method according to claim 2, wherein said encoding block comprises at least one bi-encoder, in particular a multilayer perceptron, step (b1) comprising, for each comparable reference object, obtaining a representation of said comparable object, by applying said encoding block to both the first and / or second vector(s) of the candidate object, and to the first and / or second vector(s) of said comparable reference object.

4. A method according to any one of claims 2 and 3, wherein the attention block implements a dot product calculation, step (b2) comprising, for each comparable reference object, obtaining the weight of said comparable reference object, by the dot product of the representation of said comparable reference object and the representation of the candidate object.

5. A method according to any one of claims 2 to 4, wherein said prediction neural network comprises a single encoding block and a single attention block, the encoding block being applied to both the first and second vectors of the candidate object and comparable reference objects.

6. A method according to any one of claims 1 to 5, wherein step (a) comprises the selection: - a first set of reference objects comparable to said candidate object in terms of structural characteristics, by application at least to the first vector of said candidate object of a first information retrieval model trained on said database; and - a second set of comparable reference objects to said candidate object in terms of geographical characteristics, by applying at least to the second vector of said candidate object a second information retrieval model trained on said database.

7. A method according to any one of claims 1 to 6, wherein the information retrieval block(s) and the prediction neural network form a Retrieval-Enhanced Machine Learning, REML, type set.

8. A method according to any one of claims 1 to 7, wherein said structural characteristics of the object are not geographical.

9. Method according to claim 1 to 8, wherein said attention block comprises a first auxiliary block, step (b2) comprising the adjustment, for each comparable reference object, of the weight of said comparable reference object by application of said first auxiliary block.

10. A method according to any one of claims 1 to 9, wherein said predictive neural network further comprises a second auxiliary block, step (b2) comprising, for each comparable reference object, the adjustment of the known value of the target quantity for said comparable reference object by application of said second auxiliary block.

11. A method according to any one of claims 1 to 10, wherein the predictive neural network comprises an aggregation layer at the output of the attention block, the method comprising a step (b3) of computing a weighted sum of the known values ​​of the target quantity for each comparable reference object, by said weights of each reference object comparable, said raw aggregation value, by application of said aggregation layer; the value of said target quantity for said candidate object being estimated either directly as said raw aggregation value, or as a function of said raw aggregation value.

12. Method according to claim 11, wherein said prediction neural network further comprises at least one decoding block downstream of the attention block, step (b) comprising a step (b-4) of adjusting said raw aggregation value by applying said decoding block to a so-called global representation of the candidate object.

13. A method according to any one of claims 11 and 12, wherein said decoding block comprises a multilayer perceptron and a security mechanism using a bounded function, typically hyperbolic tangent, step (b-4) comprising the calculation of an adjustment factor to be applied to the raw aggregation value.

14. A method according to any one of claims 1 to 13, comprising a step (aO) of training, by data processing means (21) of a second server (2), the information retrieval model and said prediction neural network, from said reference object database.

15. First server (1) for predicting the value of at least one target quantity exhibiting a geographical dependency for a candidate object defined by a first vector of structural characteristics of said object and a second vector of geographical characteristics of said object, characterized in that it comprises data processing means (11) configured to: - Select, from a database of reference objects stored on data storage means (12) accessible by said first server (1), each reference object also being defined by a first vector of structural characteristics of said object of 31 reference and a second vector of geographical characteristics of said reference object, and associated with a known value of said magnitude, at least one set of reference objects comparable to said candidate object, by application to the first vector and / or the second vector of said candidate object of at least one information retrieval model trained on said database of reference objects; - Estimate the value of said target quantity for said candidate object as a function of the known values ​​of said quantity for the reference objects comparable to said candidate object selected, by application of a prediction neural network comprising at least one attention block, including obtaining a weight of each comparable reference object according to a proximity of the first vector of the comparable reference object with the first vector of the candidate object and / or a proximity of the second vector of the comparable reference object with the second vector of the candidate object.

16. Product computer program comprising code instructions for the execution of a method according to any one of claims 1 to 14 of predicting the value of at least one target quantity exhibiting a geographical dependence for a candidate object defined by a first vector of structural features of said object and a second vector of geographical features of said object, when said program is executed on a computer.

17. Computer-readable storage means on which is recorded a computer program product comprising code instructions for the execution of a method according to any one of claims 1 to 14 of predicting the value of at least one target quantity exhibiting a geographical dependence for a candidate object defined by a first vector of structural characteristics of said object and a second vector of geographical characteristics of said object.

Images