Computer-based monitoring method for detecting a failure state in a pavement joint

Abstract

Computer-implemented monitoring method for detecting a failure state of a pavement joint. The invention relates to a computer-implemented monitoring method for detecting a failure state of a pavement joint (1), comprising the steps of: obtaining at least one monitoring data point from at least one audible signal generated when at least one vehicle (V) passes over said joint (1), referred to as the "monitoring signal"; analyzing the monitoring data using a machine learning model previously trained to recognize the failure state of pavement joints (1); and providing information representative of the failure state of said joint (1). Figure for the abstract: Figure 8

Description

Title of the invention: Computer-based monitoring method for detecting a failure state in a pavement joint. Technical field

[0001] The present invention relates to engineering structure equipment and more specifically to road joints.

[0002] In particular, the invention relates to a computer-implemented monitoring method for detecting a failure state of a pavement joint.

[0003] The invention also relates to a computer-implemented method for learning a pavement joint classification model. Technological background

[0004] A road joint is used to connect two sections of road surface that may undergo relative movement. This joint is arranged to fill the gaps extending between the sections of pavement concerned, thus ensuring continuity of movement.

[0005] A pavement joint is installed in pavement structures, such as roads, highways, bridges, or airport runways, to absorb movements due to thermal variations, mechanical loads, vibrations, and other external forces. This joint helps prevent cracking, deformation, and structural damage that could result from these movements.

[0006] Among the known road joints, we distinguish among others the gap joints ([Fig. aa]), the cantilever joints ([Fig.lb]), or the modular joints (figures le).

[0007] By its function, the joint plays a crucial role in allowing relative movements between the different sections of the pavement, as well as in absorbing thermal and mechanical stresses.

[0008] However, due to its constant exposure to traffic loads and climatic variations, the pavement joint is subject to various types of degradation, such as cracks, chips or deformations.

[0009] These degradations can affect different components of the joint.

[0010] For example, these can appear at the level of the flashing 3 of the joint, as illustrated in the example in [Fig. 2a]. In particular, the flashing can lose its adhesion to the surface layer due to wear or degradation of the interface between the constituent materials. The flashing materials can also deteriorate over time under the effect of excessive loads, UV radiation, and climatic conditions.

[0011] The joint anchors can also suffer deterioration caused by various factors, such as corrosion in the presence of water, de-icing salts or other chemicals, or plastic deformation due to excessive loads. An example of damage affecting the anchors is illustrated in [Fig. 2b].

[0012] The sealing elements, such as the metal plates 5, can also deteriorate. Examples of such deterioration are illustrated in [Fig. 2c].

[0013] Another example of a disorder that can affect the proper functioning of the pavement joint is relative transverse displacement, as illustrated in [Fig. 2d]. This refers to the relative lateral or horizontal movement between two adjacent pavement segments, perpendicular to the main direction of traffic. This movement can result from various forces and conditions, such as traffic loads, thermal variations, seismic activity, or deformation of the underlying structure, such as ground movement or subsidence.

[0014] The pavement joint may also be subject to a relative elevational offset, as shown in [Fig. 2e]. This is a relative vertical displacement between two adjacent pavement segments. This movement can lead to surface irregularities and increased structural stresses.

[0015] Other problems can compromise the quality of the seal, such as sealing defects. This can lead to water infiltration, causing freeze-thaw damage and promoting corrosion, as illustrated in [Fig. 2e].

[0016] Whatever its nature, a degradation of the road joint is likely to lead to a significant degradation of the quality of the ride, requiring costly repairs and posing risks to road users.

[0017] Road joint inspection is usually carried out through visual checks and physical measurements. Such methods can be costly, subjective, and sometimes insufficiently consistent.

[0018] Indeed, visual inspections are often subject to the inspector's interpretation and may lack precision and consistency in the case of insufficiently experienced teams.

[0019] Physical measurement techniques, on the other hand, may require traffic interruptions, thus increasing costs and inconvenience for users.

[0020] There is therefore a need for a method allowing proactive and regular monitoring of the condition of pavement joints in order to prevent major failures. Summary of the invention Monitoring method

[0021] The invention aims to meet this need and relates to a computer-implemented monitoring method for detecting a failure state of a pavement joint, comprising the steps of: a. obtain at least one monitoring data point from at least one audible signal generated when at least one vehicle passes over said joint, referred to as the "monitoring signal", b. analyze the monitoring data using a machine learning model previously trained to recognize the failure state of pavement joints, and in particular, in the case of supervised learning, of at least one joint of the same type as the one being monitored. c. provide information representative of the state of failure of said seal.

[0022] By "road joint" one must understand a segment of joint or several segments of joint arranged end to end and constituting a line of joints.

[0023] The invention makes it possible to take advantage of the sound signals generated by vehicles when they cross the pavement joint. Analysis of these signals makes it possible to detect characteristic variations that may indicate joint degradation. This analysis provides an objective and consistent diagnosis, thus reducing the need for manual on-site inspections.

[0024] In addition, the analysis of sound signals through a machine learning model allows for easy and precise detection of potential faults, thus ensuring a proactive and efficient response.

[0025] Proactive monitoring allows for early intervention on the road joint, thus preventing degradation from causing more serious damage affecting other elements of the roadway and posing a risk to users.

[0026] The reduction in the need for visual inspections and the prevention of major repairs also allow for substantial savings.

[0027] Furthermore, the sensor(s) used for recording sound signals interfere little or not at all with road traffic, thus avoiding service interruptions.

[0028] In addition, thanks to the invention, operators have a predictive maintenance process that allows them to make more informed decisions and plan repair work more efficiently.

[0029] The pavement joint can be of any suitable type. It can be chosen from gap joints, cantilever joints, supported joints, supported strip joints, improved lining joints and modular joints.

[0030] The term "vehicle" refers to any motorized land vehicle, whether it is a means of transport for persons and / or goods or a construction vehicle. It may be equipped with wheels, tracks, or a combination of these. It can also be a dedicated inspection vehicle, with two or more axles. Monitoring signal acquisition

[0031] The signal can be acquired by means of one or more fixed sensors on the structure, in which case the noise generated by road traffic passing over the joint is recorded, or by means of one or more mobile sensors, carried by one or more inspection vehicles traveling over the structure, in which case the noise generated by the rolling of these vehicles as they pass over the joint is recorded. In the latter case, the inspection vehicle(s) can periodically travel over the joint(s) to be diagnosed.

[0032] Preferably only one sensor is used per joint line, but several sensors can still be used per joint line, in particular when there is a noise barrier between traffic lanes or when the number of traffic lanes is large and the sensors can be placed under the road.

[0033] The method may include a step of recording the monitoring signal using at least one fixed sensor located near the pavement joint, said recording step having, particularly in the case of a fixed sensor on the structure, a duration preferably greater than 30 minutes, and more preferably greater than 1 hour. In the case of a mobile sensor on an inspection vehicle, the recording may be of shorter duration, for example limited to the time the inspection vehicle spends traveling on the structure over a predefined distance before and after the joint to be monitored. Geopositioning of the inspection vehicle may be used to automatically trigger the start and end of the recording.

[0034] When fixed, the sensor can be placed on the surface, in particular on one side of a traffic lane, preferably opposite said joint.

[0035] Preferably, the active part of the sensor is oriented towards the road joint.

[0036] Alternatively, the sensor is at least partially buried, in particular arranged in a cavity or in a conduit under the pavement joint.

[0037] Alternatively, the sensor is placed at a height relative to the road joint, in particular suspended above the road joint, for example on a bridge or a high structure such as a pole or a sign.

[0038] The sensor can be arranged to record sound signals continuously or at specific intervals.

[0039] When present on an inspection vehicle, the sensor can be positioned outside the vehicle, for example under the rear bumper, avoiding placement under an axle to prevent interference with the noise due to the Doppler effect caused by the vertical movement of the axle at the joint. In this case, it is preferable to record not only the noise passing over the joint but also the vehicle's position. in real time, so as to identify the joint more easily as well as the speed of the vehicle in order to characterize the data more precisely.

[0040] In certain embodiments, in order to reduce the size of the recordings, recording is started upon detection of a triggering event whose sound level exceeds a predefined threshold. When threshold triggering is used, it may be advisable to start recording the sound signal a few moments before this event to avoid any loss of relevant information.

[0041] For this purpose, the sensor can be equipped with a pre-trigger function. The sensor is then configured to maintain an active buffer. When the triggering event is identified, the records prior to and following this event are saved.

[0042] The sensor may include storage means, for example a hard drive or an SD card. An operator can thus retrieve the records stored in said storage means in order to transmit them to a record processing unit.

[0043] Alternatively, the sensor includes said processing unit.

[0044] Alternatively or additionally, the sensor may include a wireless data transmission system to transmit the recordings to said processing unit.

[0045] The sensor may include a power supply unit, preferably a battery, in particular a rechargeable one.

[0046] An example of a sensor compatible with the invention is that marketed by Zoom Corporation under the name "Zoom H2n".

[0047] The sensor may include a satellite geolocation system providing information on its position.

[0048] The sensor may include a protective grid or housing to protect it against intrusions and climatic conditions. Monitoring signal processing

[0049] The method may include, prior to step a), the step of processing the monitoring signal to obtain the monitoring data, said processing step comprising segmenting the monitoring signal into a plurality of first discrete segments of predefined time duration, said duration being preferably less than 2 hours, better less than 1 hour, and even better less than 30 minutes. In the case of a mobile sensor, the segmentation of the monitoring signal is limited to the duration of the passage at the joint, for example, to a duration on the order of a second.

[0050] Segmenting the monitoring signal reduces the amount of data that needs to be processed simultaneously. The segments can be processed in parallel, which This allows the use of distributed computing architectures or multi-core processing units, thereby increasing processing efficiency. Furthermore, by working on reasonably sized segments, the memory required at any given time is reduced. This prevents memory overflows, particularly for systems with limited resources.

[0051] The aforementioned processing step may further include filtering the first segments so as to attenuate in said segments at least one component relating to at least one event other than the passage of the vehicle, said component having a signal amplitude below a first predetermined threshold. The filtering may be carried out using a technique for reducing or eliminating background noise from a sound recording, such as the "spectral noise gating" technique. For more information concerning this technique, those skilled in the art are invited to refer to the article "Sainburg, T., Thielk, M., & Gentner, TQ (2020). Finding, visualizing, and quantifying latent structure across diverse animal vocal repertoires. PLoS computational biology, 16(10), el008228."

[0052] The first threshold mentioned above may correspond to the number of standard deviations beyond the mean to apply the filter, and its value may be greater than 1.5, better than 2, for example between 2 and 3.

[0053] The monitoring signal processing step prior to step a) may consist of processing the monitoring signal so as to detect in the monitoring signal a signature characteristic of the passage of the vehicle over the joint, possibly after filtering it, in particular a signature characteristic of the passage of the axles of the same vehicle over the road joint, typically the presence of at least two peaks of amplitude greater than a given threshold and close in time, i.e. separated temporally by less than a predefined duration, and the segmentation of the signal around this characteristic signature.

[0054] The monitoring signal processing step may therefore include: - the identification in the first segments, particularly filtered ones, of at least one characteristic signature corresponding to the passage of the vehicle, said characteristic signature including in particular one or more peaks of amplitude greater than a second predefined threshold, - the generation of at least a second segment of reduced size corresponding to a portion of the first segment containing said characteristic signature.

[0055] The second threshold may be greater than 4%, or even 10%, of the maximum amplitude observed in the first segment.

[0056] The characteristic signature can be defined by the amplitude of the peaks and by the minimum distance between two successive peaks as mentioned above. The latter This parameter allows grouping several peaks exceeding the second threshold into a single event if they are sufficiently close, as is frequently observed for the various axles of a vehicle, for example. This enables particularly efficient segment selection.

[0057] The minimum distance between two successive peaks may be less than 3 times the size of the second segment.

[0058] The width of the second segment can be between 0.8 s and 2.5 s. An extended width implies more laborious processing without guaranteeing a proportional increase in relevant information, while a width that is too narrow risks obscuring part of the data, particularly during prolonged passages of trucks compared to those of cars.

[0059] Advantageously, the monitoring signal processing step may further include obtaining a time-frequency domain representation of a signal segment containing the useful information, in particular of the second segment and / or the first segments, said representation comprising a plurality of points, each point linking a signal amplitude value to a frequency value and a time value.

[0060] The representation may have a resolution between 128*128 points and 512*512 points.

[0061] The time-frequency representation can be obtained by applying a short-time Fourier transform (STFT).

[0062] Alternatively, the time-frequency representation is a mel-spectrogram obtained by: - Application of a time-frequency transform, in particular a short-term Fourier transform on the corresponding segment, followed by - A frequency interpolation to convert the linear Hertz scale into a linear mels scale.

[0063] Alternatively, the time-frequency representation is obtained by applying a wavelet transform.

[0064] The wavelet transform can be a Synchrosqueezed Wavelet Transform (SSWT). The operation of such a transform is detailed in "Muradeli, J. "ssqueezepy GitHub." (2020)".

[0065] The wavelet transform can be a continuous wavelet transform. Further details on the method can be found in the document “Carpine, R. (2022). Detection of defects in civil engineering structures by wavelet analysis of vibration response (Doctoral dissertation, Gustave Eiffel University)”.

[0066] The processing step may further include reducing the dimension of said time-frequency representation so as to obtain a smaller representation.

[0067] The reduced-size time-frequency representation refers to a simplified version of the initial data in the time-frequency domain. This reduction allows the information in the representation to be condensed along specific directions determined by a projection method. This approach reduces the complexity of the initial representation while preserving the main characteristics of its temporal and frequency variations.

[0068] This dimensionality reduction may involve the implementation of principal component analysis (PCA). Principal component analysis (PCA) involves identifying the principal directions of variation in the time-frequency space and projecting the data from the representation onto a lower-dimensional space, thus enabling a significant reduction in complexity while preserving the fundamental structure of the corresponding sound signal.

[0069] Alternatively or additionally, dimensionality reduction may involve implementing linear discriminant analysis (LDA). This analysis aims to identify directions in the time-frequency space that maximize the separation between different classes or categories of data, and to project the data onto these discriminant directions. This makes it possible to reduce dimensionality while preserving relevant discriminant information.

[0070] Advantageously, the monitoring data corresponds to the time-frequency representation or to the reduced-size time-frequency representation. Machine learning

[0071] Learning can be supervised or unsupervised. Unsupervised learning can be useful for monitoring the degradation state of a seal for which there is no pre-existing model, but for which one wishes to monitor a possible degradation over time, relative to a reference state.

[0072] Classification model

[0073] Preferably, step b) of monitoring data analysis involves providing said monitoring data, in particular in the form of a time-frequency representation, to a pavement joint classification model trained so as to assign to an input data a class among a plurality of failure classes and / or subclasses, said plurality of classes preferably comprising a first class of "joints in good condition" associated with pavement joints exhibiting a functional state, and a second class of "joints in poor condition" associated with pavement joints exhibiting a defective state.

[0074] The plurality of classes may also include a third class "joints in average condition" associated with pavement joints exhibiting an intermediate condition.

[0075] The classes "joints in poor condition" and "joints in average condition" can also be assigned one or more subclasses characterizing the type of failure, such as "degraded flashing" or "anchoring defect".

[0076] The information needed to label the training data is obtained, for example, when an operator collects sensor recordings. The operator can provide details on the condition of the joint for which the recordings were made. This information may be included among those required on a specific form. Other information may also be recorded, such as the type of pavement joint, its coordinates, the date of recording and / or collection of said recording, the condition of the joint, the type of damage (if any), the type of sensor used, and its position relative to the joint.

[0077] The method may include a step of training the classification model by machine learning, the training step comprising: - obtaining a plurality of training data from audible signals generated when a plurality of vehicles pass over one or more reference road joints, referred to as "training signals", said training data being distributed across said classes and / or subclasses of failure state, a label indicating membership in one of the classes and / or subclasses being associated with each training data point, and - the implementation of supervised training of a classification model from the training data and said labels of belonging to one of the classes and / or subclasses in order to obtain said trained pavement joint classification model.

[0078] The reference pavement joint(s) may be chosen from among gap joints, cantilever joints, or modular joints, among others.

[0079] Preferably, the road joint and the reference road joint(s) are of the same type.

[0080] Supervised training of the classification model can be carried out using a machine learning algorithm.

[0081] Preferably, the machine learning algorithm is a decision tree algorithm.

[0082] The decision tree algorithm can be a random forest algorithm (“RF” for Random Forest).

[0083] In a preferred embodiment, the decision tree algorithm is an Extreme Gradient Boosting (XGBoost) type algorithm such as, for example, described in the document "Chen, T., & Guestrin, C. (2016, August). XGBoost: A scalable tree boosting System. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining (pp. 785-794)”.

[0084] The number of decision trees used by the algorithm can be less than 500, better less than 300, even better less than 200, in particular in the order of 100.

[0085] Each decision tree can have a depth of less than 60, better less than 50, even better less than 30, in particular on the order of 10.

[0086] In a variant concerning the fixed sensor, the learning is unsupervised. The learning is then associated with the evolution of the state of a road joint, rather than with a type of joint. The objective is to detect any drift in the sound signature over time, which is a possible indicator of damage.

[0087]

[0088] Acquisition of training signals

[0089] The acquisition of the drive signals can be done in the same way as the acquisition of the monitoring signal, in particular when the sensor(s) are fixed relative to the joint.

[0090] The drive signals are thus, in examples of implementations of the invention, obtained using sensors located in the vicinity of the reference pavement joints, fixed

[0091] The method may include a step of recording the drive signals, said recording step preferably having a duration greater than 30 min, better greater than Ih, when the sensor(s) are fixed, as for the acquisition of the monitoring signal.

[0092] As mentioned previously, the sensor(s) can be placed at various locations near the reference pavement joints. When fixed, the nature and location of the sensor(s) during the acquisition of the drive signals can be the same as during the acquisition of the monitoring signal.

[0093] In the case of a mobile sensor on an inspection vehicle, the learning phase can be carried out with one or more fixed sensors, and the monitoring phase with a mobile sensor on the inspection vehicle. Preferably, processing is performed to obtain a match between the sound signatures obtained by the mobile sensor and those obtained by the fixed sensor, including filtering and scaling. Training signal processing

[0094] Preferably, the training data is obtained by processing each training signal. The training signal processing step is preferably similar, in particular identical, to that described for the monitoring signal.

[0095] The processing may thus include segmenting the training signal into a plurality of first discrete training segments of predefined time duration, said duration being preferably less than 2 hours, better less than 1 hour, even better less than 30 minutes.

[0096] Preferably, the aforementioned processing includes filtering the first training segments so as to attenuate in said segments at least one component relating to at least one event other than the passage of vehicles, said component having a signal amplitude below the first threshold. The filtering can be carried out using the "spectral noise gating" technique.

[0097] In the same way as for the monitoring signal, the processing of the training signal may include: - the identification in the first training segments of at least one characteristic signature corresponding to the passage of the vehicle, said characteristic signature comprising, for example, one or more peaks of amplitude greater than the second threshold, and - the generation of at least one second training segment of reduced size corresponding to a portion of the first training segment having said characteristic signature.

[0098] As mentioned above, the processing step may further include obtaining a time-frequency domain representation of the second segment(s) of the drive signal, said representation comprising a plurality of points, each point linking a signal amplitude value to a frequency value and a time value. The representation may have a resolution between 128*128 and 512*512.

[0099] The processing step may further include reducing the dimension of said time-frequency representation so as to obtain a smaller representation.

[0100] Said dimensionality reduction may involve the implementation of a principal component analysis (PCA) or a linear discriminant analysis (LDA), as explained previously.

[0101] Advantageously, the training data corresponds to the time-frequency representation or the reduced-size time-frequency representation of the second training segment.

[0102] Information representative of the failure state

[0103] The information representing the state of failure delivered in step c) preferably includes the class and / or subclass to which the pavement joint belongs.

[0104] Information representing the failure state can be displayed on a display device, in particular on a computer, mobile phone or tablet screen. This information can be stored in a file with the seal identifier. Learning process

[0105] The invention further relates, independently or in combination with the foregoing, to a computer-implemented method for learning a road joint classification model, the method comprising the steps of: - obtaining a plurality of training data from sound signals generated during the passage of a plurality of vehicles over several reference joints, referred to as "training sound signals", said training data being distributed over a plurality of failure classes and / or subclasses, said plurality of classes and / or subclasses comprising a first class of "joints in good condition" associated with road joints exhibiting a functional state, and a second class of "joints in poor condition" associated with road joints exhibiting a defective state, a label of belonging to one of the classes and / or subclasses being associated with each training data, - to carry out supervised training of a classification model from the training data and said labels of belonging to one of the classes and / or subclasses in order to obtain the road joint classification model allowing to assign to an input data a class and / or subclass among the classes and / or subclasses of the plurality of classes and / or subclasses of failure state.

[0106] Preferably, the supervised training of the classification model is carried out using a machine learning algorithm.

[0107] Preferably, the machine learning algorithm is a decision tree algorithm, in particular of the XGboost type.

[0108] Preferably, the learning method according to the invention incorporates all the features previously mentioned in relation to step b) of analysis of the monitoring method according to the invention.

[0109] As mentioned previously, the training data is obtained by processing each training signal, said processing comprising segmenting the training signal into a plurality of first discrete training segments of predefined time duration, said duration preferably being less than 2h, better less than 1h, even better less than 30 min.

[0110] The processing may include filtering the first training segments so as to attenuate in said segments at least one component relating to at least one event other than the passage of vehicles, as defined above.

[0111] The first threshold can be as defined above.

[0112] The training signal processing step may include the detection of a characteristic signature in the training signal, optionally after filtering, in particular a characteristic signature of the passage of the axles of the same vehicle over the joint, typically the presence of at least two amplitude peaks close in time, i.e separated temporally by less than a predefined duration, and the segmentation of the signal around this characteristic signature.

[0113] The treatment may therefore include: - the identification in the first training segments of at least one characteristic signature corresponding to the passage of the vehicle, said characteristic signature comprising, for example, one or more peaks of amplitude greater than a second predetermined threshold, and - the generation of at least one second training segment of reduced size corresponding to a portion of the first training segment having said characteristic signature.

[0114] The second threshold is preferably as defined above in connection with the monitoring signal processing.

[0115] The characteristic signature can be defined by the amplitude of the peaks and by the minimum distance between two successive peaks, as defined above, as well as the minimum distance between two successive peaks and the width of the second segment.

[0116] The processing may further include obtaining a time-frequency domain representation of the second training segment(s), said representation comprising a plurality of points, each pixel linking a signal amplitude value to a frequency value and a time value.

[0117] The time-frequency representation can be obtained by applying a short-time Fourier transform (STFT),

[0118] Alternatively, the time-frequency representation is a mel-spectrogram or is obtained by applying a wavelet transform, in particular a wavelet synchrosqueezing transform as mentioned above or a continuous wavelet transform (CWT).

[0119] The training signal processing step may further include reducing the dimension of said time-frequency representation so as to obtain a reduced size representation.

[0120] Said dimensionality reduction may involve the implementation of a principal component analysis (PCA) or a linear discriminant analysis (LDA).

[0121] The training data preferably corresponds to the time-frequency representation or to the reduced-size time-frequency representation.

[0122] Reference pavement joints can be selected from gap joints, cantilever joints, supported joints, supported strip joints, improved lining joints or modular joints.

[0123] The reference pavement joints used to train the classification model preferably belong to the same type of joint as that being monitored.

[0124] The drive signals are preferably obtained using sensors located in the vicinity of the reference pavement joints, as mentioned above.

[0125] The invention also relates to a method for maintaining a road joint comprising the steps of: - to implement the monitoring method according to the invention, - to intervene on the road joint in case of failure of said joint in order to repair or replace the latter.

[0126] The invention further relates to a computer program product, comprising code recorded on a physical medium or downloadable from a server, comprising code instructions intended to be executed on computer equipment such as a computer, tablet or mobile phone, these instructions when executed leading to the implementation of the monitoring method according to the invention, and / or the learning method according to the invention.

[0127] The invention further relates to a monitoring system for detecting a failure state of a pavement joint, comprising - at least one sensor, in particular permanently installed in the vicinity of the road joint or on an inspection vehicle, arranged to record at least one audible signal generated when at least one vehicle passes over said joint or when said inspection vehicle passes over said joint - a processing unit for the sound signal(s) recorded by the sensor(s), said unit being configured to implement the monitoring method according to the invention.

[0128] The processing unit may include a computer or a computing server.

[0129] The processing unit may include computing means comprising one or several processors, microprocessors or microcontrollers as well as storage means in which a computer program product is stored, in the form of a set of code instructions executable by the processor(s).

[0130] The processing unit can be located at the network edge (“edge computing”), i.e. as close as possible to the sensor.

[0131] The sensor may have all or some of the characteristics described above. Brief description of the figures

[0132] The following description, with reference to the accompanying drawings, given by way of non-limiting examples, will clearly explain what the invention consists of and how it can be implemented. In the accompanying figures:

[0133] [Fig. la], [Fig.lb] and [Fig. le] Figures la, 1b and le are examples of pavement joints;

[0134] [Fig.2a], [Fig.2b], [Fig.2c], [Fig.2d], [Fig.2e] [Fig.2f] Figures 2b, 2c, 2d, 2e, and 2f illustrate examples of deteriorated road joints;

[0135] [Fig.3] The [Fig.3] is an example of data acquisition according to the invention;

[0136] [Fig.4] Fig.4 is a block diagram illustrating different stages of a process learning a model for classifying road joints, according to the invention;

[0137] [Fig.5] Fig.5 is an example of a sound signal processing step according to the invention;

[0138] [Fig.6] Fig.6 is an example of training the classification model according to the invention;

[0139] [Fig.7a], [Fig.7b], [Fig.7c], [Fig.7d], [Fig.7e] Figures 7a, 7b, 7c, 7d, 7e are examples of data obtained according to the invention;

[0140] [Fig.8] Fig.8 is a block diagram illustrating different stages of a process monitoring to detect a failure state of a pavement joint, according to the invention, and

[0141] [Fig.9] Fig.9 is an example of a time-frequency representation obtained according to the invention. Description of method(s) of implementation

[0142] In the figures, and unless otherwise specified, identical elements bear the same reference symbols.

[0143] Figure 3 shows an example of data acquisition for use in the various processes according to the invention, as will be described later.

[0144] As illustrated, the passage of a vehicle V over a road joint 1 generates sound waves W. The waves thus generated are captured and transformed into electrical signals by at least one sensor 10 located in the vicinity of the road joint 1.

[0145] The sensors 10 can have various locations on the structure, as illustrated in [Fig.3]. It is also possible to have one or more sensors mounted on an inspection vehicle.

[0146] When the sensor(s) 10 are fixed, the sensor(s) 10 can, for example, be placed on the surface, in particular on one side of a traffic lane, opposite said joint 1, as represented by references P5 and P3 in [Fig. 4]. In the example shown, position P5 is located behind an obstacle O.

[0147] The sensor(s) 10 can be placed under the seal 1, in particular inside a cavity or conduit. The sensor(s) can, for example, be placed under a bridge inside the deck, which corresponds to reference P2, or outside the deck, designated by PI in [Fig. 4].

[0148] The sensor(s) 10 can also be placed at a height relative to the road joint, in particular suspended above the road joint, as illustrated by reference P4 in [Fig.4].

[0149] The recording of sound signals can be carried out continuously or at specific intervals. In particular, when the sensor(s) are fixed, the duration of the recordings can be greater than 30 minutes, preferably greater than 1 hour, for example on the order of 2 hours.

[0150] An example of a sensor compatible with the acquisition according to the invention is marketed by Zoom Corporation under the name "Zoom H2n".

[0151] To minimize recording size, the sensor(s) 10 can be configured to start recording only upon detection of a triggering event whose sound level exceeds a predefined threshold. When threshold triggering is used, and to prevent any potential loss of relevant information, the sensor 10 can be configured to maintain an active buffer. When the triggering event is identified, the recordings before and after that event are saved. Such a function is called pre-triggering.

[0152] The sensor 10 may include storage means, for example a hard drive or an SD card. This allows an operator to retrieve the records stored in said means in order to transmit them to a processing unit 20. The sensor may include a power supply unit, preferably a battery, in particular a rechargeable one.

[0153] When the sensor(s) are mounted on an inspection vehicle, they are placed, for example, under the rear bumper, avoiding placing them under an axle so as not to disturb the noise by the Doppler effect due to the vertical movement of the axle at the joint.

[0154] Advantageously, the sensor or each sensor 10 can be housed in a casing to be protected against intrusions and climatic conditions.

[0155] The sound signals recorded by the sensor(s) are then transmitted to the processing unit 20 for processing according to the methods of the invention which will be described below

[0156] Figure 4 illustrates an example of a method for learning a pavement joint classification model.

[0157] This classification model will then be used to evaluate the failure state of pavement joints 1, as will be described later.

[0158] The method first comprises a phase A aimed at obtaining a plurality of training data associated with reference pavement joints 1 whose state of The defect is known. Pavement joints 1 can correspond to any type of joint, as previously mentioned.

[0159] Advantageously, the reference pavement joints belong to the same type of pavement joints as those that will be monitored subsequently.

[0160] The training data comes from a plurality of sound signals generated when a plurality of vehicles V pass over the reference pavement joints, also referred to as "training sound signals" in what follows.

[0161] Phase A first includes a step 110 of acquiring the drive signals, as described previously with reference to [Fig.3], using one or more sensors 10 arranged in the vicinity of the reference pavement joints 1.

[0162] The recorded training signals are then processed in step 120 by the processing unit 20 to obtain the aforementioned training data.

[0163] Processing step 120 will be detailed with reference to [Fig. 5]. Step 120 includes a substep 121 of segmenting the sound signals into a plurality of first training segments Se; of reduced duration, in particular on the order of 10 min in the illustrated example. For further details on the segmentation method used, those skilled in the art may refer to the article “Boche, H., & Mônich, UJ (2019). Downsampling of bounded bandlimited signals and the bandlimited interpolation: Analytic properties and computability. IEEE Transactions on Signal Processing, 67(24), 6424-6439”.

[0164] Next, each segment is subjected to a filtering process aimed at reducing the level of unwanted background noise, this process corresponding to substep 123. This filtering of the first segments Se 7a therefore aims to reduce, in these segments, at least one component related to at least one event other than the passage of vehicles, this component having a signal amplitude below a first predetermined threshold. The filtering or denoising can be carried out using any known filtering technique. A particularly suitable example of an algorithm is that designated as "spectral noise gating".

[0165] Preferably, the first threshold is described as the number of standard deviations above the mean for applying the filter, with a value that can be greater than 1.5, better than 2, for example between 2 and 3

[0166] The processing step 120 further includes a substep 125 for detecting the passage of vehicles in the first segments thus filtered.

[0167] This detection involves identifying, in the first segments Se / mentioned above, at least one characteristic signature Sc corresponding to the passage of the vehicle. This characteristic signature Sc comprises one or more peaks Pki; Pk2 with an amplitude greater than a second threshold. This signature Sc can be defined by the amplitude of the peaks as well as by the minimum distance between two successive peaks. This latter parameter allows several peaks exceeding the second threshold to be grouped into a single event if their proximity is sufficient, as is often the case for the different axles of a vehicle, for example.

[0168] Preferably, the second threshold is greater than 4%, or even 10% as mentioned above.

[0169] The minimum distance between two successive peaks can be greater than 3 times the size of the second segment.

[0170] Substep 125 further comprises the generation of at least a second training segment Se 2, corresponding to a portion of the first segment containing said characteristic signature.

[0171] The processing step 120 further includes a substep 127 in which a time-frequency representation Re of the second segments is generated according to a predefined resolution px1 * px2. An example of such a representation Re is shown in [Fig. 9]. This representation Re comprises a plurality of points, each point linking a signal amplitude value to a frequency value and a time value.

[0172] Preferably, pxl and px2, corresponding respectively to the number of points in each of the two dimensions, are identical.

[0173] Substep 127 generation of a time-frequency representation can implement a short-time Fourier transform (STFT).

[0174] Alternatively, substep 127 implements a wavelet transform.

[0175] The wavelet transform can be a wavelet synchrosqueezing transform or a continuous wavelet transform (CWT).

[0176] Alternatively, the representation generated in substep 127 is a mel-spectrogram obtained by: - Application of a short-term Fourier transform on the corresponding segment, followed - An interpolation of frequencies to convert the linear scale of Hertz into a linear scale of mels.

[0177] The processing step 120 further includes a substep 129 of reducing the dimension of said time-frequency representation R so as to obtain a reduced size representation.

[0178] Said dimensionality reduction may involve the implementation of a principal component analysis (PCA) or a linear discriminant analysis (LDA) as mentioned above.

[0179] Thus, at the end of phase A, we obtain the training data which are used to obtain a classification model according to the invention.

[0180] The training data are distributed over a plurality of failure classes and / or subclasses.

[0181] Said plurality of classes and / or subclasses includes in the illustrated example a first class Cl of "joints in good condition" associated with pavement joints exhibiting a functional state. Figure 7a is an example of the average of the time-frequency representations obtained for joints of class CL

[0182] The plurality of classes and / or subclasses further includes a second class C2 of "joints in poor condition" associated with pavement joints exhibiting a defective condition. An example of the average time-frequency representations obtained for joints of class C2 is illustrated in [Fig. 7b].

[0183] Figure 7c illustrates the difference between the two representations in Figures 7a and 7b. As can be seen, defective joints generate a higher noise level. The frequency peak of defective joints tends to shift towards higher frequencies.

[0184] Figure 7d illustrates the difference between the average time-frequency representations obtained for C2 class joints and those obtained for the subclass of joints in poor condition with degraded flashing. The frequency peak of joints with degraded flashing tends to show secondary peaks just before and just after the main peak.

[0185] Figure 7e illustrates the difference between the average time-frequency representations obtained for C2 class joints and those obtained for the subclass of joints in poor condition and with an anchoring defect. The frequency peak of joints with an anchoring defect tends to shift towards higher frequencies.

[0186] A label of belonging to one of the classes and / or subclasses, in particular to one of the classes Cl and C2, is associated with each training data obtained in phase A.

[0187] The information needed to label the training data is obtained, for example, when the operator collects the recordings from the sensor(s) 10. The operator can provide details on the condition of the joint for which the recordings were made. This information may be included among those required in a specific form. Other information may also be recorded, such as the type of pavement joint, its coordinates, the date of recording and / or collection of said recording, the condition of the joint, the type of damage (if any), the type of sensor used, and its position relative to the joint.

[0188] The training data obtained at the end of phase A are separated into a training data set DE and a validation data set Dv.

[0189] Phase B of the process is illustrated in detail in [Fig.6].

[0190] The training data belonging to the classes and / or subclasses, in particular to the classes Cl and C2, are separated into a training data set DE and a validation data set Dv.

[0191] By way of example, training data may correspond to training data collected before a predefined date, while validation data may correspond to that collected after that date.

[0192] Phase B includes a training step 130, here supervised, of a classification model.

[0193] Supervised training can be performed by various algorithms known for this purpose. In the illustrated example, learning is implemented using an XGBoost type decision tree algorithm trained on the DE training data.

[0194] The algorithm constructs decision trees to determine the class of an input data point. Each tree attempts to predict the class of the individuals in a different way. The final result is obtained by summing the results of all the trees. The classifier can be parameterized as follows: - Number of trees: This parameter controls the number of trees that will be built during training. This number can, for example, be around 100 trees. - Tree depth: This parameter sets a maximum size for decision trees. The depth can, for example, be 10.

[0195] Optionally, phase B includes a validation step. The trained classification model is evaluated using a validation dataset by calculating a prediction score on the training data and a prediction score on the validation data. This evaluation aims to verify that the difference between the two scores is not excessive. If this difference is too large, retraining can be performed by adjusting the algorithm parameters.

[0196] Once the classification model has been trained and validated, it is used to predict the failure state of a pavement joint 1.

[0197] In one variant, the learning is unsupervised and can be carried out as follows. Filtering, passage detection and extraction, time-frequency representation, and information reduction by PCA and / or LD A are performed identically to supervised learning. Based on the processed recording data, data clusters are automatically formed, for example, by implementing a K-means, DBSCAN, or Mean Shift type algorithm. It is preferable to record data over a period of at least one week to encompass data that take into account daily temperature variations, and preferably between 3 and 9 months to obtain Data taking into account seasonal climatic variations. After this recording phase, the damage to a pavement joint is deduced from the evolution of at least one of these clusters.

[0198] A computer-implemented monitoring method for detecting a failure state of a pavement joint according to the invention is described below with reference to [Fig.8].

[0199] Similar to the learning process of [Fig.4], the monitoring process includes a first phase A' of obtaining monitoring data from at least one sound signal generated when at least one vehicle V passes over said joint 1, also called the "monitoring signal".

[0200] This phase A' can be similar to phase A of process signal acquisition when the sensor(s) 10 are fixed.

[0201] Thus, phase A' may include an initial step 210 of acquiring at least one monitoring signal, as described in [Fig.3], using a sensor 10 located in the vicinity of the reference pavement joint 1.

[0202] In the case where the sensor(s) are mobile, the acquisition is carried out using one or more sensors on board the vehicle, for example positioned under the rear bumper of the vehicle.

[0203]

[0204] The recorded sound signals are then processed in step 220 by the processing unit 20 to obtain the monitoring data.

[0205] The monitoring signal processing step 220 can be similar to that of the process in [Fig.4] in the case of fixed sensors 10, and is detailed in [Fig.5].

[0206] Step 220 may thus include a substep 211 of segmenting the sound signal into a plurality of first segments of reduced duration, in particular on the order of 10 min in the illustrated example.

[0207] Next, a filtering substep 223 can be applied to each first segment in order to reduce obsolete information. Also, during this filtering substep, at least one component relating to at least one event other than the passage of vehicles is attenuated in the first segments, said component having a signal amplitude below a first predetermined threshold.

[0208] The processing step 220 also includes a substep 225 for detecting vehicle passages in the first filtered segments. This detection consists of identifying in these segments at least one signature characteristic of a vehicle's passage. This signature is defined by one or more peaks with an amplitude greater than a second threshold, as defined above. The signature can be characterized by the amplitude of the peaks as well as by the minimum distance between two successive peaks. This latter parameter makes it possible to group several peaks exceeding the second threshold into one. only event, if their proximity is sufficient, as frequently occurs for the different axles of a vehicle.

[0209] Substep 225 further comprises the generation of at least a second segment, corresponding to a portion of the first segment containing said characteristic signature.

[0210] The processing step 220 further includes a substep 227 in which a time-frequency representation of the second segments is generated according to a predefined resolution.

[0211] Substep 227 of generating a time-frequency representation may implement a short-term Fourier transform or a wavelet transform, in particular a synchrosqueezing wavelet transform (SSWT) or a continuous wavelet transform (CWT).

[0212] Alternatively, the representation generated in substep 227 is a mel-spectrogram.

[0213] The processing step 220 further includes a substep 229 for reducing the dimensionality of said time-frequency representation R so as to obtain a reduced size representation. Said dimensionality reduction may include the implementation of a principal component analysis (PCA) or a linear discriminant analysis (LDA).

[0214] Thus, at the end of phase A', the monitoring data is obtained to be evaluated in order to determine the failure state of the corresponding pavement joint 1.

[0215] In the case of one or more mobile sensors, the same phase A' applies, where the parameters of the same type of filtering can be adapted to specifically reduce the relative importance of rolling noise and airflow.

[0216] To determine the failure state of a joint, the method includes a phase B' of analysis of the monitoring data using a machine learning model previously trained to recognize the failure state of pavement joints.

[0217] Phase B' includes a step 230 in which said monitoring data is provided to the pavement joint classification model trained according to the method of [Fig.5] so as to assign to an input data a class and / or subclass from among the plurality of failure classes and / or subclasses mentioned above.

[0218] Preferably, the classification model is trained with data from reference pavement joints of the same type as the pavement joint.

[0219] After phase B' of analysis, the process delivers, in step 240, information representative of the failure state. Preferably, such information includes the class to which the pavement joint belongs.

[0220] Information representing the state of failure can be displayed on a display device, in particular on a computer, mobile phone or tablet screen.

[0221] The invention is not limited to the examples just described.

[0222] Other machine learning algorithms can be used. For example, other machine learning algorithms can be considered, such as random forests (RF), k nearest neighbors (KNN), stochastic gradient descent (SGD), logistic regression (LOGR), support vector machine (SVM).

[0223] The filtering of sound signals can be achieved using other techniques.

[0224] The information representing the failure state of the seal may also include the probability of the road joint belonging to the class to which the associated monitoring data has been assigned.

Claims

Demands

1. A computer-implemented monitoring method for detecting a failure state of a pavement joint (1), comprising the steps of: a. obtaining at least one monitoring data from at least one sound signal generated when at least one vehicle (V) passes over said joint (1), referred to as the "monitoring signal", b. analyzing the monitoring data using a machine learning model previously trained to recognize the failure state of pavement joints (1), and in particular of at least one joint of the same type as that which is being monitored, c. delivering information representative of the failure state of said joint (1).

2. A method according to claim 1, comprising prior to step a), step (220) of processing the monitoring signal so as to obtain the monitoring data, said processing step (220) comprising segmenting (221) the monitoring signal into a plurality of first discrete segments of predefined time duration, said duration being preferably less than 2h, better less than 1h, even better less than 30 min.

3. Method according to the preceding claim, the processing step (220) further comprising filtering (223) the first segments so as to attenuate in said segments at least one component relating to at least one event other than the passage of the vehicle (V), said component having a signal amplitude less than a first predetermined threshold.

4. A method according to any one of the preceding claims, comprising prior to step a), step (220) of processing the monitoring signal so as to detect in the monitoring signal a signature characteristic of the passage of the vehicle over the joint, optionally after filtering it, and of segmenting the signal around this characteristic signature.

5. A method according to the preceding claim, said signature being characteristic of the passage of the axles of the same vehicle over the road joint, and corresponding to the presence of at least two peaks of amplitude greater than a given threshold and separated temporally by less than a predefined duration.

6. A method according to claim 2 or 3 on the one hand, and one of claims 4 and 5 on the other hand, the processing step comprising: - the identification in the first segments of said at least one characteristic signature corresponding to the passage of the vehicle, said characteristic signature comprising in particular one or more peaks of amplitude greater than a second predefined threshold, - the generation of at least one second segment of reduced size corresponding to a portion of the first segment comprising said characteristic signature, in particular a portion substantially centered around the characteristic signature.

7. A method according to any one of the preceding claims, the processing step comprising obtaining a time-frequency domain representation of a segment of the monitoring signal comprising the sound information useful for generating the monitoring data, in particular of the second segment(s) as defined in claim 6, said representation comprising a plurality of points, each point linking a signal amplitude value to a frequency value and a time value.

8. A method according to the preceding claim, the processing step further comprising reducing the dimension of said time-frequency representation so as to obtain a reduced-size representation, said dimension reduction preferably comprising the implementation of a principal component analysis (PCA).

9. Method according to claim 7 or 8, the monitoring data corresponding to the time-frequency representation or to the reduced-size time-frequency representation.

10. A method according to any one of the preceding claims, comprising a step (210) of recording the monitoring signal using at least one sensor (10) disposed in the vicinity of the pavement joint, said recording step preferably having a duration greater than 30 min, better greater than 1h.

11. A method according to any one of claims 1 to 9, said vehicle (V) being an inspection vehicle, and said at least one sensor being disposed on this vehicle.

12. A method according to any one of claims 1 to 10, said at least one sensor being fixed relative to the pavement.

13. A method according to any one of the preceding claims, monitoring data analysis step b) comprising supplying said monitoring data to a pavement joint classification model trained to assign to an input data a class from among a plurality of failure classes and / or subclasses, said plurality of classes and / or subclasses comprising a first class (C1) of "joints in good condition" associated with pavement joints exhibiting a functional condition, and a second class (C2) of "joints in poor condition" associated with pavement joints exhibiting a defective condition.

14. Method according to the preceding claim, the class "joints in poor condition" comprising at least one subclass, in particular a subclass corresponding to a degraded flashing or a defective anchorage.

15. A method according to claim 13 or 14, comprising a machine learning training step for the classification model, the training step comprising: - obtaining a plurality of training data (De) from sound signals generated during the passage of a plurality of vehicles over reference road joints, referred to as "training signals", said training data being distributed over said classes and / or subclasses (Cl; C2) of failure states, a label of belonging to one of the classes and / or subclasses being associated with each training data, and - carrying out supervised training of a classification model from the training data and said class membership labels so as to obtain said trained road joint classification model.

16. Method according to the preceding claim, the supervised training of the classification model is carried out using a decision tree algorithm.

17. A method according to any one of claims 15 and 16, wherein the training data (TD) is obtained by processing each training signal, said processing comprising: - segmenting (121) the training signal into a plurality of discrete first training segments (Se1) of predefined time duration, said duration preferably being less than 2 hours, better less than 1 hour, even better less than 30 minutes. - Optionally, filtering (123) the first training segments (Se1) so as to attenuate in said segments at least one component relating to at least one event other than vehicle passages, said component having a signal amplitude below the first threshold.- the identification in the first training segments of at least one characteristic signature (S c) corresponding to the passage of the vehicle (V), said characteristic signature (Sc) including for example one or more peaks (Pki ; Pk2) of amplitude greater than the second threshold, - the generation of at least one second training segment (Se 2) of reduced size corresponding to a portion of the first training segment including said characteristic signature (Sc), in particular a portion substantially centered around the characteristic signature (Sc).

18. Method according to the preceding claim, the processing step further comprising obtaining (127) a time-frequency domain representation (Re) of the second drive segment(s) (Se 2), said representation comprising a plurality of points, each point relating a signal amplitude value to a frequency value and a time value.

19. A method according to the preceding claim, the processing step further comprising reducing (129) the dimension of said time-frequency (Re) representation so as to obtain a reduced size representation, said dimensionality reduction preferably involving the implementation of a principal component analysis (PCA).

20. Method according to claim 18 or 19, the training data being the time-frequency representation (Re) or the reduced-size time-frequency representation of the second training segment.

21. Method according to any one of claims 15 to 20, in step c) the information representative of the failure state comprising the class and / or subclass to which the pavement joint belongs.

22. A method according to any one of the preceding claims, wherein the reference pavement joint and / or pavement joints are selected from gap joints, cantilever joints, supported joints, supported strip joints, improved coating joints or modular joints, preferably, the pavement joint to be monitored and the reference pavement joints are of the same type.

23. A method for maintaining a road joint comprising the steps of: - implementing the monitoring method according to any one of the preceding claims, - intervening on the road joint in the event of failure of said joint so as to repair or replace the latter.

24. A computer-implemented method for training a pavement joint classification model, the method comprising the steps of: - obtaining a plurality of training data (TD) from sound signals generated when a plurality of vehicles pass over reference joints, referred to as "training sound signals," said training data being distributed over a plurality of failure classes and / or subclasses, said plurality of classes and / or subclasses comprising a first class (C1) of "joints in good condition" associated with pavement joints exhibiting a functional state, and a second class (C2) of "joints in poor condition" associated with pavement joints exhibiting a defective state. Failure state classes, a label membership in one of the classes is associated with each training data point. - to carry out supervised training of a classification model from the training data and said labels of belonging to one of the classes and / or subclasses in order to obtain the classification model of road joints allowing to assign to an input data a class among the classes (Cl; C2) and / or subclasses of the plurality of classes and / or subclasses of failure states.

25. Method according to the preceding claim, the supervised training of the classification model being carried out by means of a decision tree algorithm.

26. A method according to claim 24 or 25, wherein the training data (TD) is obtained by processing each training signal, said processing (120) comprising: - the segmentation (121) of the training signal into a plurality of first discrete training segments of predefined time duration, said duration being preferably less than 2h, better less than 1h, even better less than 30 min. - Optionally, filtering (123) the first training segments (Sel) so as to attenuate in said segments at least one component relating to at least one event other than the passage of vehicles (V), said component having a signal amplitude lower than a first predetermined threshold, - the identification (125) in the first training segments of at least one characteristic signature corresponding to the passage of the vehicle (V), said characteristic signature comprising for example one or more peaks (PI; P2) of amplitude greater than a second predetermined threshold, - the generation of at least one second training segment (Se2) of reduced size corresponding to a portion of the first training segment containing said characteristic signature.

27. ​​A method according to the preceding claim, the processing step further comprising: - obtaining a time-frequency domain representation (Re) of the second drive segment(s), said representation comprising a plurality of points, each point linking a signal amplitude value to a frequency value and a time value, and - optionally, reducing the dimension of said time-frequency representation so as to obtain a reduced-size representation, said dimension reduction preferably comprising the implementation of a principal component analysis (PCA).

28. Method according to any one of claims 24 to 27, the training data corresponding to the time-frequency representation or to the reduced-size time-frequency representation.

29. Product computer program, comprising code recorded on a physical medium or downloadable from a server, comprising code instructions intended to be executed on computer equipment such as a computer, tablet or mobile phone, these instructions when executed leading to the implementation of the monitoring method according to any one of claims 1 to 23, and / or the learning method according to any one of claims 24 to 28.

30. A monitoring system for detecting a failure state of a road joint (1), comprising - at least one sensor (10), in particular permanently located in the vicinity of the road joint, arranged to record at least one sound signal generated when at least one vehicle passes over said joint, - a processing unit (20) for the sound signal(s) recorded by the sensor(s), said unit being configured to implement the monitoring method according to any one of claims 1 to 22.

Images