Eddy current sensor inspection system with AI

Abstract

The invention relates to an eddy current sensor inspection system with an AI. A method for detecting and measuring a metal object (2, 2a-2c), such as a reinforcement embedded in a concrete building structure (54), in which electromagnetic sensor signals (S, Se) are emitted and inductive response signals (S, Sr) are measured at different sensing positions (x) of a surface (55) of the structure (54), thereby measuring the sensing positions (x) in such a way that it is determined that the response signals (S, Sr) are position-determined. The detection of the objects (2, 2a-2c) and the determination of their depth (z) and / or diameter (d) are based on an AI model evaluating the response signals (S, Sr).

Description

Technical Field

[0001] The present invention generally relates to a method and a corresponding system for detecting and measuring objects embedded in a concrete building structure. Background Art

[0002] Nondestructive testing (NDT) techniques play an important role in monitoring and diagnosing construction work with embedded structures. An example of an embedded structure is reinforced concrete, which is a heterogeneous material in which concrete is poured around a network of reinforcing members (such as steel reinforcements) in order to increase the ability to carry or withstand tensile stresses that might otherwise cause cracking.

[0003] In particular, the detection and characterization of the properties of embedded metal objects such as reinforcing steel or bars (or often abbreviated rebar) in a surrounding medium such as a concrete element is crucial for quality control during the construction phase, and for example for the inspection of unknown structures (e.g. to avoid hitting the rebar when drilling), or for health monitoring in the form of regular inspections and post-disaster safety assessments. The parameters of the objects that need to be inspected include in particular their position or two-dimensional location and depth (cover thickness or overburden) as well as diameter.

[0004] NDT technology is described, for example, in EP 2159605 A1, which involves ground penetrating radar, ie a system that sends high frequency (>10 MHz) electromagnetic waves to a target and records the time of flight in the material.

[0005] On the other hand, documents such as US 4837509 A or US 6541965 B1 relate to electromagnetic based inspection techniques.A benefit of electromagnetic systems is that they are generally relatively insensitive to the material properties of embedded materials such as the concrete itself and other non-metallic materials that may be embedded in the concrete in addition to the steel bars.

[0006] Therefore, it is known in the art to use inductive sensors (such as eddy current sensors) to sense embedded rebars. Such detectors typically work by measuring changes in selected parameters as a user sweeps the sensing head across a surface of interest (such as a concrete wall or floor). Through the sweeping motion of the detector, the location and length or orientation of the hidden object (e.g., rebar) can be determined by using the received response signals.

[0007] Eddy currents are generated in conductive materials in response to a suitable time-varying magnetic field applied to the conductive material. The time-varying magnetic field exerts a force on the electrons in the conductive material, creating an electric current (called "eddy current"). The eddy currents themselves generate an electromagnetic field (called an induced magnetic field) that opposes the incident magnetic field. Eddy current sensors include a magnetic coil that excites the time-varying magnetic field toward the concrete and receives an induced secondary magnetic field from the conductive object. When inspecting conductive or magnetizable buried objects, especially metal objects such as rebar, the induced secondary magnetic field is sensitive to both the rebar diameter and the depth. However, it is difficult to accurately obtain these two parameters in a direct manner, especially simultaneously.

[0008] Known eddy current systems (such as those described in US 5339023 A or US 5446379 A) apply one or several harmonic voltages to a coil to generate a time-varying magnetic field within a metallic test piece. This magnetic field induces eddy currents in nearby electrical conductors, which weaken the source magnetic field. The weakening of the source magnetic field produces a measurable voltage signal in a nearby pickup coil, which contains information about the electrical, magnetic and geometric properties of the test piece.

[0009] However, with currently known eddy current sensor based NDT systems and methods, accurate detection of embedded objects and in particular accurate determination of object diameter and overburden thickness remains challenging, in particular differentiation of objects that are close to each other. Summary of the invention

[0010] It is therefore an object of the present invention to improve eddy current based systems and methods for detecting objects embedded in concrete building structures.

[0011] It is a further object to provide an improved eddy current based system and method for sensing the position and diameter of such embedded objects.

[0012] The present invention therefore relates to a method for detecting and measuring metal objects such as reinforcing bars, metal (e.g. copper, iron or steel) pipes or prestressed cables embedded in a concrete building structure such as a currently constructed or completed building, facility or infrastructure project. The method has the steps of emitting an electromagnetic sensor signal and measuring an induced response signal at different sensing locations on the surface of the construction structure, whereby the sensing locations are measured in such a way that the response signal is position-determined, so that the response signal is associated with the location of the sensing path (respectively the surface).

[0013] Furthermore, there is the step of determining the object positioning as the object position and the object diameter and / or the depth relative to the surface by evaluating the response signal, wherein the object is detected and the position of the object is determined by at least one artificial intelligence (AI) model, such as one or more neural networks, as examples of deep learning models or "classical" machine learning (ML) models, such as support vector machines (SVM), random forests or decision tree algorithms.

[0014] For example, one part of the AI ​​model is implemented as a "detection" neural network for detecting an object and determining the location of the object, and another part of the AI ​​model is implemented as a regression neural network for determining the depth and / or diameter of the detected object. Thus, machine learning (ML) artificial intelligence (AI) is used to locate the object.

[0015] Thus, features are extracted from the response signal by filtering the signal and / or using an algorithm to determine the signal derivative. These features provide input to the AI ​​model in the form of a feature vector along with the response signal itself. Feature extraction is performed, for example, by a neural network such as a convolutional neural network.

[0016] Thus, the input (i.e., at least the feature vector) is the sensed positions discretized by discrete sensed position increments of a predetermined size (e.g., with equal size or equidistant position steps). Objects are detected relative to these position increments and the position and depth and / or diameter of the object are determined. Thus, for such position intervals or segments of the scanned surface, the presence or absence of an object is determined based on a response signal assigned to or associated with the position interval, e.g., where the interval length or range is in the millimeter range.

[0017] As an option, the features extracted from the response signal are first or second order derivative values ​​of the values ​​of the (particularly low pass filtered) response signal.

[0018] As another option, the evaluation of the response signal is completed using a sliding window method. Thus, for each window, a detection of the AI ​​model is performed and / or the window shift amount is determined by the position increment size.

[0019] In a further development of the method, there is also the determination of the reinforcement diameter and / or depth by means of a regression neural network and / or by means of a multi-class classifier.

[0020] Additionally or alternatively, there is also a measurement of at least one further sensor signal besides the inductive sensor signal and an evaluation of the further signal for determining the object position, depth and / or diameter.

[0021] Alternatively or additionally, an AI model (such as a detection neural network and / or a regression neural network and / or a multi-class classifier) ​​can be automatically configured based on sensed or stored pre-known data or manually configured by a user (e.g., setting parameters), and / or an AI model (which is best suited for a given measurement task characterized by, for example, its structural structure and / or object properties) can be selected from a variety of available AI models.

[0022] As another example of another signal, the system includes a ground penetrating radar (GPR) that is used to measure and provide a ground penetrating radar signal that is similar to the sensing signal referenced with respect to a surface or sensing location. Therefore, the ground penetrating radar signal provides another input to the "sensing" AI model, or is input into another "GPR" AI model, which can be referred to as low-level fusion (LLF). In the latter case, detecting an object and determining the object location, depth and / or diameter is accomplished by fusing the outputs of two AI models (the "sensing" AI model and the "GPR" AI model) (e.g., a detection neural network and / or a regression neural network or a multi-class classification neural network).

[0023] The ground penetrating radar (respectively the GPR signal) can also be used to classify the sensed embedded objects based on both the sensing response signal and the ground penetrating radar signal, since GPR is sensitive not only to metal or sensing objects (such as iron or steel bars), but also to non-metallic objects, for example. Thus, this evaluation of different types of sensor signals can be used to detect and distinguish different types of hidden objects or voids within a structure.

[0024] As another option, at least a third input to the AI ​​model provided by stored information about the construction structure and / or object (e.g., stored nominal or previously measured rebar diameters), the stored information being retrieved, for example, from a Building Information Model (BIM) comprising the construction structure. As another example, such input may be done manually by a user. If the diameter of an object, such as a rebar diameter, is known, this may be used as an input to, for example, deep regression or classification, and / or selection of a specific neural network trained specifically for that diameter.

[0025] In some further developments of the method, the detection of objects such as steel bars is based on a predefined detection probability threshold, which serves as a criterion for classifying the output of an AI model (e.g., the above-mentioned detection neural network) as "detected" or "not detected".

[0026] As a further development, there is the determination of a central position (such as the middle or the centre of gravity) of a position interval associated with the detected object and the determination of the object depth and / or diameter from the central position.

[0027] Optionally, the detection neural network and the regression or multi-class classification neural network each comprise a convolutional neural network (CNN) and a fully connected network (FCN), wherein the respective fully connected network is fed with the output of the respective convolutional neural network.

[0028] The invention also relates to a system for detecting and measuring metal objects embedded in a concrete building structure. The system comprises an eddy current sensor for emitting an electromagnetic sensor signal and measuring an induced response signal at different sensing locations on the surface of the building structure and a position sensor for measuring the sensing locations in such a way that the response signal is position determined (i.e. the position of the response signal along a measurement trajectory or within a scanning area is known).

[0029] The system further comprises a control and evaluation unit for determining the object location as the object position and the object diameter and / or the depth relative to the surface by evaluating the response signal. Thereby, the control and evaluation unit is configured to detect objects and determine their position and the diameter and / or the depth of the objects using the AI ​​model of the system (e.g., a detection neural network for object detection and object position determination, and a regression or multi-class classification neural network of the system to further determine the depth and / or diameter of the detected objects).

[0030] The control and evaluation unit is further configured to algorithmically extract features from the response signal by signal filtering and / or determining signal derivatives and to provide a feature vector comprising at least the response signal and the extracted features.

[0031] The control and evaluation unit is further configured to discretize inputs regarding the sensing positions by discrete sensing position increments, which are predefined and optionally of equal size, and to detect objects relative to the position increments and determine their position and depth and / or diameter.

[0032] In a further development, the system comprises a user interface, and the control and evaluation unit is configured to retrain at least one of the AI ​​models (e.g. a detection neural network and / or a regression or multi-class classification neural network) based on user input via the user interface (in particular corrections such as confirmation or non-confirmation of an object position, depth and / or diameter determined by the system).

[0033] In another further development, the eddy current sensor comprises a coil arrangement of a pair of connected coils (preferably with a circular or rectangular cross-section) surrounded by a separate third coil (preferably with an elliptical or rectangular cross-section), wherein all coil axes are arranged perpendicular to the measuring surface so that the magnetic field can enter the measuring surface essentially perpendicular to the surface.

[0034] In a system with such two sensing coil units, the control and evaluation unit is optionally configured to measure and evaluate the coil pair sensor signals and the single coil sensor signals in a parallel / synchronous or alternating manner (in a dual sequential sensing process for the same sensing position) to determine the corresponding diameter and / or depth respectively using the AI ​​model, i.e., to determine a diameter or depth value based on the coil pair sensor signals and to determine another diameter or depth value based on the single coil sensor signal. In the case of sequential measurement, the controller is also configured to at least temporarily store at least one of the at least two determined diameter and / or depth values ​​in a memory of the system for the final fusion of the two subsequently determined diameter and / or depth values.

[0035] The invention also relates to a computer program product having computer executable instructions for automatically executing the steps of the claimed method, in particular on a system as claimed, respectively a control and evaluation unit of the claimed system. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The systems, methods and arrangements according to the invention are described or explained in more detail below, by way of example only, and with reference to the working examples schematically shown in the accompanying drawings. The diagrams of the accompanying drawings should not be considered to be drawn to scale. Where appropriate, the same reference numerals are used for the same features or features with similar functions. Different indices of the reference numerals are used to distinguish between different embodiments of the same or equivalent features shown exemplarily.

[0037] Specifically,

[0038] Figure 1 Examples of sensing devices, systems, and methods for detecting embedded metal objects are shown;

[0039] Figure 2 An example of using artificial intelligence (AI) to detect and measure embedded metal objects is shown;

[0040] Figure 3a to Figure 3d An example of detection by a recurrent neural network is shown;

[0041] Figure 4 Yet another example of using artificial intelligence (AI) to detect and measure embedded metal objects is shown;

[0042] Figure 5 An example of the results of object detection and depth / diameter estimation as output of the AI ​​model is shown;

[0043] Figure 6 A method for training a detection neural network and a recurrent neural network is shown;

[0044] Figure 7a , Figure 7b An example of further development of an object sensing system and method that fuses multiple sensor data is shown;

[0045] Figures 8a to 8c An example of a coil arrangement for a sensing device is shown; and

[0046] Figures 9a to 9c Shows the use of Figures 8a to 8c Example of a sensing device using artificial intelligence (AI) to detect and measure embedded metal objects. DETAILED DESCRIPTION

[0047] Figure 1 An example of a device 50 which can be part of a detection or measurement system and a method for detecting metal objects embedded in a concrete structure, such as a reinforcing bar 2, which will be mentioned below as a preferred but not exclusive example of such an embedded metal object, is shown. In this example, steel bars 2, which partially have different diameters d (see steel bars 2a in contrast to steel bars 2b, 2c), are laid at different locations in a concrete wall 54. To detect the steel bars 2, the eddy current sensor device 50 moves the steel bars 2 at a defined (e.g. constant) or measured speed v. x Scanning over a surface 55 of a wall 54 defines a sensing trajectory or displacement direction x starting from a zero sensing position x0.

[0048] The detection of the steel bar 2, in particular the determination of the position of the steel bar 2 (i.e. its x position and depth z) is based on the induction principle, so the sensor device 50 comprises an eddy current sensor 51 with a sensor coil 52 for emitting an electromagnetic signal Se. The signal Se pervades the wall up to a certain penetration depth z. max , and the sensor receives a response signal Sr from the conductive material (such as rebar steel 2). The response signal Sr is evaluated by the system processor for sensor control and data processing / signal evaluation (not shown). Figures 8a to 8c A more detailed example of a sensor coil for transmitting and receiving signals is given in .

[0049] In other words, the measurement of the signal amplitude of the sensor 51, such as a voltage U, is also used to detect the rebar position x and the depth z towards the scanning surface 55 of the detected rebar and the diameter d of the rebar. In addition to the eddy current measurement in volts, the displacement or sensing position x measured in the scanning direction starting from the starting sensing position x0 of the measuring device 50 is also used as a measured variable. The displacement measurement does not need to be limited to one dimension as shown in this simple example, but can also be done in two directions. Furthermore, multiple 1D scans / line scans can be combined to cover a 2D area or provide a 2D map of the rebar structure 2.

[0050] In this example, the x displacement is determined by odometer in the form of measuring a rotation R using a rotation sensor (not shown) on a wheel or axle of the scanning device 50. Due to the movement in the scanning direction x, the wheel turns, which is measured by the rotation sensor (in units of scale). These scales are converted into displacements (for example in millimeters). Other options for measuring the change in position of the sensor 50 are known to the skilled person, for example based on imaging and using algorithms such as SLAM (Simultaneous Localization and Mapping) or using an IMU (Inertial Measurement Unit). Therefore, the sensor 50 can include one or more cameras for positioning the sensor 50 on the wall, wherein these cameras can also be used to measure the displacement of the sensor 50.

[0051] The eddy current sensor 51 measures the response Sr (in volts) of the conductive material (in this case, the reinforcing steel 2) according to the magnetic field Se injected at a specific frequency. In the figure, an example of the response signal Sr as the original signal caused by electromagnetic induction due to the presence of three reinforcing steel bars 2a, 2b, 2c is shown. The received signal Sr shows three signal peaks, which are more or less different due to the relatively dense or crowded arrangement of the reinforcing steel bars 2. In such a dense or crowded arrangement of the reinforcing steel bars, the received signal peaks are difficult to distinguish. In such complex scenarios, the signal processing-based methods known in the art usually fail, while the AI / ML-based method proposed in this article overcomes such defects by training from a large number of examples.

[0052] These time measurements are interpolated and discretized by the control and evaluation unit of the detection system (respectively) the sensor 50, ie converted into a displacement discrete signal Sd of equidistant distance values ​​Δx in volts (see also Figure 2 )(spatial discretization). The signal is cut into windows 1 of a predefined length L with a constant number N of values. Thus, the measurement data are collected according to the so-called sliding window method. Window 1 is shifted, skipping one signal value each time. The discretization in the distance direction thus determines the step width of the window 1 shift or the sensing position increment Δx (in millimeters), for example, the discretization step width and its window shift amount equal to 0.25 mm, 0.5 mm, 1 mm, 2 mm or 4 mm.

[0053] The data is passed to a detector that performs rebar detection and estimates the depth and diameter of the rebar. The detector can be part of the sensor device 50 or located remotely. For rebar detection, rebar location and rebar property determination, an artificial intelligence (AI) model-based approach is used, as described in the context of the following figure.

[0054] In this example, each of the subsequent windows produces an execution of the AI ​​model, which can provide real-time signal evaluation to provide immediate feedback to the user on possible detection and regression or multi-class classification. Alternatively, the entire signal time series can be evaluated at once, for example, combining sensor signals to form an image such as a measurement result, which can ultimately be output, for example, as a 2D plot as described above.

[0055] Figure 2 An example of using artificial intelligence (AI), respectively a measurement system with a machine learning algorithm such as a neural network, to detect and measure embedded objects such as rebar 2 is shown.

[0056] In this example, we first perform a spatial step Δx (see Figure 1 ) is filtered. Thus, a (more) smoothed signal Sf is extracted by passing the signal Sd through a low-pass filter. Furthermore, in this example, the first-order derivative Sf' and the second-order derivative Sf" of the filtered signal Sf are calculated and used as additional inputs. These four variables Sd, Sf, Sf' and Sf" are used as inputs 3 of a first AI model, which in this example is a first neural network (named as a detection network DN).

[0057] Based on the input variables Sd, Sf, Sf', Sf" (these input variables are combined in a feature vector as input 3), and using training data (the AI ​​system is trained for scenarios where the steel bar configuration is known), the detection neural network DN estimates whether there is a steel bar 2 in the discrete position segment Δx. Therefore, the output 4 of the detection network is a binary result in the following form: "steel bar is detected" with respect to the segment Δx (where the square "2" in the figure symbolizes the first output neuron), or "steel bar is not detected" (where the square "2" in the figure symbolizes the second output neuron). If it is considered that a steel bar is detected in the segment Δx, the position x corresponding to the segment is taken as the position x of the steel bar 2 or a part of the steel bar 2.

[0058] If the output 4 of the discrete position is positive "2", in the next step, the regressive neural network RN is used to estimate the steel bar characteristics with the variables Sd, Sf, Sf', Sf" as input 3 of the regression network RN. Therefore, when the detection network DN detects the steel bar 2, the regression of the depth z of the steel bar from the scanning surface and the diameter d of the steel bar is triggered, wherein the artificial neural network RN can be similar to the structure of the detection network DN. Similarly, the output or final network layer 5 here shows two neurons, one neuron outputs the depth z of the detected steel bar, and the other neuron outputs the diameter d of the steel bar.

[0059] For the above 2D methods, the fully convolutional neural network can have a 2D map or image as output. These output images can be in the form of heat maps, where more heat indicates (higher probability of) detecting the rebar. Regression of the depth and diameter of the rebar can also be encoded in the image. Alternatively, the network DN, RN is combined with a neuron having a combined detection, regression and / or classification output.

[0060] Additional inputs to the neural networks DN, RN may be used if available or feasible, such as the displacement velocity of the sensor ( Figure 1 v x ), or known steel bar properties in advance. Such a priori known data as input to the AI ​​system can improve the estimation of the unknown properties of the steel bars. For example, if the diameter of the steel bars is considered to be known in advance (e.g., from the building plan), and only the location of the steel bars needs to be estimated, the diameter can be used as input to the AI ​​model so that the relevant contribution of the diameter to the measurement signal is taken into account.

[0061] As a further option, additional sensors on the sensor device 50 (see Figure 1 ) or information fed into the neural network DN, RN can be used to enrich the input feature vector 3 and / or apply context-specific models towards more accurate detection and prediction. For example, design information from a building information model (BIM) about the structure and materials of the wall and the properties and possible locations of buried objects (particularly the rebar 2 itself) can be used together with the known location of the equipment to extend the feature vector 3 with such data. A specific example is to feed the known (and possibly location-specific) electromagnetic material properties of the wall into the feature vector 3. As another example, a camera attached to the measurement device 50 can be used to classify the wall type and therefore provide information about the wall type as part of the feature vector 3, and / or apply subsequent ML models specific to the wall type for detection and prediction. In addition, the user can manually specify or restrict such wall or rebar properties.

[0062] As an alternative to estimating the rebar diameter d using a regression neural network RN, a multi-class classifier can be applied, especially in cases where these rebar characteristics can only take values ​​from a predefined list of discrete values ​​(e.g., standard, industry-specific rebar diameters).

[0063] The neural networks DN, RN, respectively the methods for detection and regression / multi-class classification / rebar property estimation are explained in more detail in the following figures.

[0064] Figure 3a to Figure 3dAn example of a detection network DN or a regression network RN is shown in more detail as an example of an AI model. In order to detect a steel bar as an exemplary embedded metal object based on the input 3 from the windowed and extracted features Sd, Sf, Sf', Sf", an artificial neural network DN, RN is used, each of which has two parts CNN and FCN, respectively CNN1 / CNN2 and FCN1 / FCN2.

[0065] like Figure 3a As shown, the first part CNN1 of the detection network DN in this example is a convolutional neural network (CNN), and the second part FCN1 in this example is a stack of fully connected network (FCN) layers 8.

[0066] The input signal values ​​of CNN1 are convolved with a 1D filter. The number of convolutions can vary with the number of 1DCNN layers stacked on each other. For padding, for example, a "same" method is used, in which zeros are added to the boundaries of the signal to achieve outputs of the same size. As an alternative to using zeros, the first and last values ​​of the window can also be mirrored to the outside. Other identical padding modes are also possible. The amount of value added to the window depends on the size of the filter, for example, a size of 3 results in adding one value, and a size of 5 results in the need to add two values ​​on each side to have the same padding. This method is used because for each 1D CNN layer, the previous layer is used as input and the original input is attached. Therefore, the convolution filter can learn to extract valuable features from input 3, but the original signal size is retained, thereby offsetting the compression loss caused by the CNN layer. Therefore, in this example, the activation function is variable. Pooling and / or normalization (such as batch normalization) can be optionally used, and a discard layer can also be optionally integrated.

[0067] CNN1 extracts features 6 from input features 3 (response signal Sd and its derivatives Sf, Sf', Sf') using various filters and at various layers. The CNN layers can be considered as feature extraction in addition to the smoothed signal Sf and its derivatives Sf', Sf''. The advantage of AI with artificial neural networks is that it is possible to push feature extraction into the learning domain and thus extract more valuable information from the response signal that is as uncompressed as possible (e.g., in the form of features Figure 6 in the form of ).

[0068] The output vectors / arrays 6 of the CNN part are then concatenated into a large vector 10 as the input 10 of FCN1. Again, the number of neurons in each layer, the number of layers and the type of activation function are variable. As mentioned before, the last layer 4 of the detection network DN has two output neurons, one neuron for the "2" where the rebar 2 is not detected and the other neuron for the "2" where the rebar 2 is detected, where the output 4 of the last FCN layer is aggregated by a soft-max-layer to determine the confidence of the presence of the rebar.

[0069] Figure 3b An example of a regression network RN is shown in more detail.

[0070] If the detection network DN detects a rebar 2, a regression of the depth z of the rebar from the scanned surface and the rebar diameter d is triggered. This is done by an artificial neural network RN, which has a very similar structure to the detection network DN.

[0071] The input 3 is also first processed by a variable number of 1D CNN layers as a second convolutional neural network CNN2 with a variable number of filters. After the CNN part, the network RN also includes a variable number of FCN layers 9 of a second fully connected network FCN2 as a second part with a variable size, thereby summarizing the output vector 11 of a single output feature 7 of the combined convolution part CNN2.

[0072] Unlike the detection network DN, one of the second FCN layers receives, in addition to the input 11, the sweep speed v in the entire measurement window x The abstract mean value (without scientific units) of is given as an additional input. This helps to compensate for the scan speed v x The last FCN layer 5 consists of two neurons, one outputting the depth z of the detected rebar and the other outputting the diameter d of the rebar. The actual regression output 5 of the detected rebar needs to be determined in post-processing, where the detection period is used as an additional input, as shown below in Figure 5 as further described in the context of .

[0073] Figure 3c Shows Figure 3a and Figure 3b A variation of the method described in detail in . In this embodiment, features are extracted from the input using a single feature extractor (e.g., CNN). These same features (e.g., as feature vector 10) are then fed into a classifier (e.g., FCN1) for detection and, assuming that a detection is triggered (box "2" in the figure), into a regressor (e.g., FCN2) for estimation of depth z and diameter d.

[0074] In other words, in this variation, a single feature extractor (e.g., a single CNN) is used to extract features 10 from the input 3 to feed two separate AI models FCN1, FCN2 for detection and depth / diameter determination.

[0075] Figure 3d is another embodiment. In this example, features are extracted from the input 3 using a single feature extractor (e.g., CNN). The same feature vector 10 is then fed into a single FCN that performs both detection and regression, providing an output 4 of a possible detection of "2" and an output 5 of the depth z and diameter d.

[0076] Figure 4 An alternative AI model method for feature extraction and object detection is shown. In this example, feature extraction is accomplished by a "hand-crafted" or pre- or manually designed non-machine learning feature extraction algorithm FE. The extracted features 6' are then combined again in a feature vector 10' and are used to classify "detected" / "not detected" (abbreviated as CLS in the figure) using a machine learning algorithm based on SVM or boosted decision trees, random forests, etc. The extracted features 10' are also used for regression (determining depth z and diameter d) (abbreviated as RGS in the figure), which uses an AI model such as support vector regression (SVR) or logistic regression.

[0077] That is, particularly if hand-crafted features 10 ′ are used, both object detection and object attribute determination can be accomplished using AI regression and classification techniques such as, for example, SVM or boosted decision trees.

[0078] Figure 5 An example of the results of rebar detection and depth / diameter estimation as the output of the neural network is shown, where for clarity the example is limited to one rebar rather than Figure 1 The multiple rebars in the previous example.

[0079] The upper part of the figure shows the raw signal Sr of the eddy current sensor, then below that the detection probability P, then below that the estimated depth z, and finally at the bottom the estimated rebar diameter d, all variables depending on the x-position.

[0080] Starting from the detection signal Sr, generated by the scanning of the sensor in the x-direction, thereby also covering the position of the hidden reinforcement (which gives a signal curve with a peak or maximum value as schematically depicted), and based on signal data derived from the detection signal Sr (such as the distance discretization signal, the filtered signal and the first, second or higher order derivatives Sd, Sf, Sf', Sf") as described above, by Figure 2 and Figure 3aThe detection neural network DN described in the context of determines the detection probability P.

[0081] The detection network DN first has a continuous output or detection probability curve 15 which is discretized in a subsequent step. In other words, since the detection network DN receives as input the discretized measurements of the distances in a window of a certain size and, for example, the smoothed values, their first and second derivatives Sd, Sf, Sf', Sf", the detection is not a singular peak in space, but an output 15 with increasing and decreasing confidence for a certain length 16. According to a predefined threshold 18, the probability of the detection network DN is discretized 4d, where a confidence below the threshold 18 results in "0" (i.e. not detected) and a confidence above the threshold 18 results in "1" (i.e. the rebar is detected).

[0082] For this discrete detection cycle 16 with the pattern 0-1-0, the intermediate position 20 is determined, which is then the final output 4 (in millimeters) of the rebar detection starting from the starting point of the scan in the scan direction.

[0083] The position 20 or 4 is used in the second part of the process to read as in Figure 2 and Figure 3b The depth estimation 17 and diameter estimation 19 of the regression network RN described in the context of , in order to determine the accurate depth z of the rebar from the scanning surface and the diameter d of the rebar. In other words, in order to determine the final depth value and diameter value in the depth curve and diameter curve output for the entire cycle length 16, the middle position of the binary 0-1-0 detection position "window" 16 is used as the x position value. Then, the depth value z and diameter value d corresponding to the x position value are taken as the final output of the regression network RN.

[0084] Figure 6 A method of training the detection network DN and the regression network RN, respectively, to determine the parameters of the detection network DN and the regression network RN is shown. The AI ​​rebar detection system is trained in the context of a known rebar configuration. In this case, an artificial intelligence process (AI process) in the form of a supervised machine learning process (ML process) is used, which has back propagation of error gradients to optimize the model towards a minimum difference between the predicted results and the ground truth 12, 13.

[0085] Using supervised ML, each input of the corresponding network DN or network RN requires a specified ground truth 12 or 13, i.e., the true position x of rebar 2 g or its true depth z g and diameter d gBased on this, the error of the output of the corresponding network DN, RN from the corresponding ground truth 12, 13 is used to calculate the gradient, which points to the direction of decreasing error. Then, according to the learning rate, the update is back-propagated from the output of the corresponding network DN, RN to each parameter in turn using this gradient.

[0086] In order to collect eddy current measurements and combine them with the actual reinforcement configuration x g 、z g ,d g The ground truth of the detection network DN and the regression network RN are actually learned using the training data using the above method.

[0087] Secondly, the validation set is used during training to monitor the generalization ability of the network DN, RN and prevent overfitting. Finally, the accuracy of the network DN, RN is evaluated on the final test set, which is designed in such a way that it is comparable to the training data and validation data, but the conditions are not exactly the same. Therefore, it can be guaranteed that during training, the network DN, RN will neither overfit directly to the training set nor indirectly to the validation set.

[0088] In order to make the networks DN, RN converge to good accuracy during training, several strategies are applied:

[0089] Since the above sliding window method is window 1 i Each shift of generates a detection, so the continuous confidence curve of the estimated presence of the rebar matches the ground truth x0, where the exact location of the rebar will be a singular isolated peak. However, in supervised back-propagation training of artificial neural networks, the gradient needs to point to the output of the network DN, RN and the ground truth x g 、z g or g The decreasing direction of the error between .

[0090] To solve this problem, a weighted approach is introduced, where the detection model DN and the regression model RN are not only used for the accurate reinforcement position x g There are outputs, and because the weights are small (loss weighted), the outputs before and after are not that important. The weights are distributed in a Gaussian manner at each exact bar position x g Around, as shown by symbol 14 in the figure. Therefore, loss functions such as weighted binary cross entropy loss or mean squared error loss can be applied. In addition, weighting also helps to shrink the detection peak, thereby handling crowded steel bar scenes.

[0091] As mentioned above, when rebar 2 is detected, the regression network RN is activated by the detection network DN. During the training of the network RN, the ground truth x g From the continuous sliding windows of the training regression network RN1 i Select Window 1 g (Indicated by arrow 21). The training sample is a window 1 centered at the corresponding steel bar 2. g .

[0092] In order to have not only a singular window 1 g Thus, it has a peak output for training, and provides weighting as described above, which can be used at the precise reinforcement position x g A window 1 g A certain number of windows are taken before and after (not shown in the figure for clarity). In the same way as above, the exact position x g The windows before and after have smaller weights.

[0093] Since the device can only capture a limited number of rebar configurations, the training data is always discrete in the position x, depth z and diameter d of the rebar. Therefore, there are always real settings to measure that do not appear in the training data. In order to generalize better on the test data, data augmentation is added during training to slightly change the original measurements of the sensors, so the network DN, RN does not overfit to the discrete settings of the training data.

[0094] The trained neural networks DN, RN for detection, regression and classification can be adjusted or relearned using user feedback, for example, after drilling or excavation. To this end, a feedback loop can be implemented in which the user (e.g., based on some actual drilling at the measurement location x or based on design or measurement information from another system) provides feedback on the detection and regression / classification statements of the ML model. These statements / labels can be used to retrain and / or fine-tune the ML models in order to make them more accurate overall and / or fine-tune them to a given task / location / boundary conditions, etc.

[0095] The detection system may include a display that allows visualization of the estimated reinforcement configuration (e.g., in real time) and / or be arranged at the measurement sensor or connected via a data link (e.g., a wireless connection). Such a configuration may facilitate the retraining or fine-tuning by allowing graphical user feedback or input.

[0096] In addition to user input for neural network training, the user interface can additionally or alternatively be used to allow the user to correct or confirm the detection or regression / classification results, so the neural network can be designed to output an estimate or uncertainty of the output. Low confidence values ​​in the detection and / or prediction can be used to switch from a fully automatic mode (when confidence is high) to a semi-automatic mode, in which the user is asked to check the results or is warned to be cautious of the results.

[0097] Figure 7a , Figure 7b An example of further development of a rebar sensing system and method utilizing fusion of multiple sensor data is shown.

[0098] In this case, multiple sensor types are used, for example, combining inductive sensing 51 with ground penetrating radar (GPR) 53 .

[0099] Ground Penetrating Radar is also used to detect hidden or underground assets non-invasively, so there is no need to dig or drill into embedded structures. GPS uses radar pulses to image the subsurface. When a boundary between materials with different dielectric constants (electrical susceptibility) is encountered, the transmitted radar pulse is reflected, refracted, or scattered back to the surface, where the returning radar pulse is analyzed for underground assets. For example, GPR can effectively locate plastic conduits or concrete storm drains and sanitary sewers, which cannot be effectively located using standard electromagnetic induction utility locating tools because these tools can only identify conductive materials. Like this inductive sensor, the GPR moves along a track to identify underground assets that are below the track in the area to be scanned.

[0100] The depth / diameter and material of embedded objects can be detected and determined using sensor data or a fusion of measurements derived from different measurement principles such as induction and radar. Underground objects can be made of several types of materials, such as plastic or metal. Ground penetrating radar 53 is able to detect objects regardless of their material due to its reflectivity with relatively good depth measurement capabilities and relatively low position accuracy, while electromagnetic (EM) sensors 51 are able to detect metal objects with relatively low depth measurement capabilities and relatively high position accuracy.

[0101] In accordance with Figure 7a In the example of FIG. 5 , a single measurement device 50 is used which is equipped with both an EM sensor 51 and a GPR sensor 53 . Therefore, in this example, the fusion of the sensor data is done on the sensor device 50 .

[0102] The fusion of the EM signal S and the GPR signal G is completed at the beginning of data processing, so all neural networks DN and RN act on the "EM plus GPR" data and output the "fused" results 4f and 5f. Therefore, the neural networks DN and RN act on the combined feature input vector.

[0103] In accordance with Figure 7b In the example of , separate sensor devices 50, 50' (one with EM sensor 51 and one with GPR 53) are driven on the wall in a separate case. Here, their respective sensor signals S, G are subsequently fused in a position-aware manner.

[0104] Thus, in this example, separate pipelines of EM data S and GPR data G with respective networks DN, RN and DN', RN' are used, which pipelines are unaware of each other. Thus, the sensed signal S (and respectively the features extracted therefrom) are independently fed to the first network architecture DN, RN, and the GPD signal G (and respectively the features extracted therefrom) are independently fed to the second network architecture DN', RN', as shown.

[0105] The corresponding first detection and prediction results 4, 5 and second detection and prediction results 4g, 5g of each network DN, RN and DN', RN' are finally merged into the final detection and prediction results 4f, 5f. That is, in this method, the results 4, 5 and 4g, 5g are merged into the final combined results 4f, 5f.

[0106] Different approaches to sensor fusion before or after network processing are not limited to a corresponding system of one sensor device 50 or multiple sensor devices 50, 50', but can be applied independently of this. In addition, a hybrid can be implemented, for example, two separate EM and GPR-aware neural detection network models DN and DN' first calculate candidates for detected bars 4, 4g. Then, based on the combined results 4f, two separate EM and GPR-aware neural regression networks RN, RN' predict depth / diameter estimates 5, 5g.

[0107] Moreover, in particular such detection systems with sensor fusion can be further extended to learn object type classifiers, for example based on features such as material, depth, shape, etc. Such a system can distinguish between, for example, rebars, metal pipes, non-metal pipes, metal cables, non-metal cables, mines, etc.

[0108] Figures 8a to 8c An example of a sensor coil arrangement in the form of a first sensor coil 52a and a second pair of sensor coils 52b of an eddy current sensor according to the invention is shown. Figure 8ashows a 2D overhead view of the combined coil arrangement, and Figure 8b , Figure 8c A simplified 3D perspective view is shown, where for greater clarity the sensor coil 52a is only shown in FIG. Figure 8b As shown in FIG. 5 , the coil pair 52 b is only Figure 8c Shown in.

[0109] It can be seen that the parallel arranged coil pair 52b comprises two coils shaped circularly in cross section, and the single coil 52a is shaped elliptical by two semicircles (interconnected by two lines). In addition to the coil shapes illustrated here, other coil shapes are also possible, such as rectangular shaped coils.

[0110] A single coil 52a surrounds the pair 52b. All coils 52a, 52b are arranged substantially in the same plane, with their cross-sections parallel to the surface to be measured 55. In other words, the coil axis is perpendicular to the measuring surface 55, so the (primary) magnetic field generated by the coils 52a, 52b penetrates the surface to be measured 55 substantially perpendicularly.

[0111] In this example, the winding directions of the two coils 52b (in Figure 8a The winding direction of the outer coil 52a is opposite (alternate) to that of the outer coil 52a. Compared to the single coil 52a separated from the coil pair 52b, the pair of coils 52b is electrically connected. In this example, the pair of coils 52b is connected in such a way that the current (indicated by arrows 57 in the figure) is directed in opposite directions to each other in the pair of coils 52b (however, in principle, the current can also be directed in the same direction compared to this example). As a result, as Figure 8c As shown, the magnetic field 56b of the coil pair 52b is superimposed on the common magnetic flux sensing (at Figure 8a In the 2D bird's-eye view, the magnetic flux is perpendicular to the drawing plane), while Figure 8b As shown, a single coil 52a generates a dual opposing sensing magnetic field 56a. Therefore, the eddy currents 58a and 58b generated by the magnetic fields 56a and 56b and the induced "secondary" magnetic fields 59a and 59b are perpendicular to each other, as shown in FIG. Figure 8b , Figure 8c shown.

[0112] In order to optimally sense both induced magnetic fields 59a, 59b of different orientations, the sensor may include two differently oriented pick-up coils (not shown). That is, the axis of one pick-up coil is arranged perpendicular to the surface 55 for sensing the "vertical" secondary field 59a, and the other pick-up coil is arranged parallel to the surface 55 for sensing the "horizontal" secondary field 59b. Thus, the pick-up coils may be placed side by side or nested.

[0113] The "elliptical" coil arrangement 52a has the advantage of a relatively large magnetic field spread, so that the depth can be determined relatively accurately. On the other hand, the coil pair 52b exhibits a relatively large sensitivity to the diameter of the sensing object (such as the steel bar 2), thereby allowing the object diameter to be determined relatively accurately. Therefore, combining the two in one coil arrangement can provide the advantages of both.

[0114] Figures 9a to 9c A signal evaluation method of a detection and measuring system with an eddy current sensor 50 is schematically depicted, the eddy current sensor 50 having a signal as described above for example. Figures 8a to 8c The multiple sensor coils 52a, 52b.

[0115] Figure 9a A simplified sensor device 50 is shown having a first coil 52a and a second coil 52b, wherein for clarity, only one second coil 52b is shown in addition to the first coil 52a (although Figure 8a In this example, the sensor device 50 moves in a direction F along a surface 55 of a structure embedded in an object to be detected and measured (not shown in the figure). The above principle is derived from the two sensor signals of each coil 52a and 52b (e.g. Figure 2 The inputs 3a and 3b described in the context of ) are fed into the AI ​​models DN, RN respectively. The signals or data streams of the two sources 52a, 52b are evaluated in parallel throughout the movement F.

[0116] The AI ​​model gives outputs 5a, 5b for each input 3a and 3b, the outputs 5a, 5b comprising the depth and diameter of the object as described above. The results 5a, 5b are then combined into final depth and diameter estimates 5f' (preferably in real time or synchronously) so that the measurement results are immediately provided to the user when the sensor 50 passes over the object and detects the object, as described above.

[0117] Figure 9b An alternative method is shown, in which, for clarity, Figure 9a In contrast, the process part of the method does not indicate input and AI model. In this example, the sensor device 50 moves twice on the structure surface 55, for example, once in the forward movement F and then a second time in the backward movement B, or twice in the same direction. The first pass gives a first output 5a based on the sensor signal of the first coil 52a, similar in principle to the method according to Figure 9a However, there is no parallel evaluation of the second sensor signal of the second coil 52 b at this time, since this second sensor part is inactive during the first process F and the first output 5 a is stored in the memory 60 .

[0118] The second coil 52b is not activated and the first coil 52a is inactive before the return trip B begins. Sequential activation (rather than simultaneous activation) provides the advantage that the signals (respectively the induced magnetic fields of the two coils 52a, 52b) do not overlap and do not require very challenging signal separation.

[0119] The AI ​​model, after being fed with the second sensor signal, outputs a second output 5b. The second output 5b is then preferably combined with the stored first output 5a once available to provide a final output 5f' (e.g. in the form of average depth and diameter values). It can be viewed as a set of two independent equations (due to different excitations) with two "unknowns" of depth and diameter, which can then be solved (or more accurately / reliably determined).

[0120] Thus, the individual outputs 5a and 5b may be weighted, for example, according to depth, the second depth value of coil 52b may be weighted less than the first depth value of coil 52a because, as described above, the depth estimate of coil 52a is more reliable or more accurate for relatively deeper objects.

[0121] The output 5a, 5b of each sensor part 52a, 52b (respectively, each process F, B) may include not only depth and diameter estimates, but also a vector of characteristics, which includes depth, diameter and other characteristics (in particular, the object position, the name of the underlying sensor part used (52a or 52b) and / or the sensing direction (F or B)).

[0122] Fig.9c Another evaluation method of a multi-fold eddy current sensor 50 is shown in a simplified embodiment. Figure 9b In a similar manner, two passes in the form of a forward movement F followed by a backward movement B are applied (shown in a combined view in the figure for simplicity). In each pass F or B, if Figure 9b As shown in the previous alternative method, only one coil 52a or 52b is activated or sensed in each process, which is advantageous in terms of signal processing as described above.

[0123] In this example, the outputs 5a' and 5b' of each sensor part 52a, 52b (F, B, respectively) include not only depth and diameter estimates, but the outputs 5a' and 5b' are characteristic vectors including depth, diameter and other characteristics such as object location, the name of the underlying sensor part used (52a or 52b) and / or the sensing direction (F or B).

[0124] The output vectors 5a' and 5b' are stored in the memory 60 and after the end of the last pass B (i.e. the end of the measurement, respectively) for the entire scan line or scan area, the output vectors 5a' and 5b' are read and combined into the final user output 5f' of the determined object depth and diameter.

[0125] A person skilled in the art realizes that the details shown and explained here for different entities can also be combined with details of other entities and other arrangements within the meaning of the invention.

Claims

1. A method for detecting and measuring metal objects (2, 2a-2c), in particular steel bars, embedded in a concrete building structure (54), wherein: emitting electromagnetic sensor signals (S, Se) and measuring induced response signals (S, Sr) at different sensing locations (x) on the surface (55) of the structure (54), thereby measuring the respective sensing locations (x) in such a way that the response signals (S, Sr) are position-determined, It is characterized in that extracting features (Sd, Sf, Sf', Sf", 6, 6', 7) algorithmically from said response signal (S, Sr) by signal filtering and / or determining signal derivatives, and Detecting an object (2, 2a-2c) and determining a diameter (d) and / or a depth (z) of the object relative to the surface (55) based on an artificial intelligence (AI) model, wherein the response signal (S, Sr) and the extracted features are combined in a feature vector (3, 10, 10', 11) as input to the AI ​​model; in, The feature vectors (3, 10, 10', 11) are used as input to the model comprising: discretizing the data by discrete sensed position increments (Δx) of predefined size, and An object (2, 2a-2c) is detected relative to the position increment (Δx) and the position (x) of the object (2, 2a-2c) and the depth (z) and / or diameter (d) of the object (2, 2a-2c) are determined.

2. The method according to claim 1, It is characterized in that a detection neural network (DN, DN'), as a first part of the AI ​​model, for detecting an object (2, 2a-2c) and determining the position (x) of the object, and A regression neural network (RN, RN') and / or a multi-class classification neural network (CLS), as a second part of the AI ​​model, is used to determine the depth (z) and / or diameter (d) of the detected object (2, 2a-2c).

3. The method according to claim 1 or 2, It is characterized in that Features are extracted algorithmically using neural networks, in particular convolutional neural networks (CNN, CNN1, CNN2) (6, 7).

4. The method according to any one of the preceding claims, It is characterized in that The response signal (S, Sr) is evaluated by a sliding window method, where For each window, detection is performed by the AI ​​model, and / or The offset of the window is determined by the size of the position increment (Δx).

5. The method according to any one of the preceding claims, It is characterized in that The feature vector (3, 10, 10', 11) comprises at least one additional feature provided by and / or extracted from: stored information about the building structure and / or the object (2, 2a-2c), in particular information about the building structure and / or the object (2, 2a-2c) stored by means of a building information model, or At least one further sensor signal (G) besides the inductive sensor.

6. The method according to claim 5, It is characterized in that The further sensor signal is a ground penetrating radar (GPR) signal (G), wherein the ground penetrating radar signal (G): providing at least one feature of the feature vector (3, 10, 10', 11) as input to the AI ​​model, or is input into another AI model, and the outputs of the two AI models are fused (4, 5, 4g, 5g) to complete the detection of the object (2, 2a-2c) and the determination of the object's position (x), depth (z) and / or diameter (d).

7. The method according to claim 6, It is characterized in that The sensed embedded object (2, 2a-2c) is classified based on the sensing response signal (S, Sr) and a ground penetrating radar signal (G) as another signal.

8. The method according to any one of the preceding claims, It is characterized in that As a pre-step for subsequent diameter (d) and / or depth (z) determination, the detection of the object (2, 2a-2c) is based on a predefined detection probability threshold, which serves as a criterion for classification as "detected" (2) or "not detected" (2).

9. The method according to any one of the preceding claims, It is characterized in that A center position (20), in particular a middle position or a center of gravity, of a position interval (16) associated with a detected object (2, 2a-2c) is determined, and a depth (z) and / or a diameter (d) of the object is determined based on the center position (20).

10. A method according to claims 2 and 3 or any of the preceding claims dependent thereon, It is characterized in that The detection neural network (DN, DN') and the regression neural network (RN, RN') each include a convolutional neural network (CNN, CNN1, CNN2) and a fully connected network (FCN, FNC1, FCN2), wherein the respective fully connected network (FCN, FNC1, FCN2) is fed with a respective feature vector (3, 10, 11) as the output of the respective convolutional neural network (CNN, CNN1, CNN2).

11. A system (50) for detecting and measuring objects, in particular steel bars (2, 2a-2c), embedded in a concrete building structure (54), the system comprising: an eddy current sensor (51) for emitting an electromagnetic sensor signal (S, Se) and measuring an induced response signal (S, Sr) at different sensing locations (x) on a surface (55) of the structure (54), a position sensor for measuring a corresponding sensing position in such a way that the response signal (S, Sr) is position-determined, Control and evaluation unit, It is characterized in that The control and evaluation unit is configured to detect an object (2, 2a-2c) and determine a position (x) of the object (2, 2a-2c) and a diameter (d) and / or depth (z) of the object (2, 2a-2c) based on an AI model of the system; wherein the response signal (S, Sr) provides a first input (3) of the AI ​​model, wherein the control and evaluation unit is further configured to: algorithmically extracting features (Sd, Sf, Sf', Sf", 6, 6', 7) from said response signal (S, Sr) by signal filtering and / or determining signal derivatives; Combining the response signal (S, Sr) with the extracted features (Sd, Sf, Sf', Sf", 6, 6', 7) in a feature vector (3, 10, 10', 11); discretizing the feature vector (3, 10, 10', 11) with respect to the sensing position (x) by discrete sensing position increments (Δx) of predefined size; and An object (2, 2a-2c) is detected relative to the position increment (Δx) and the position (x) of the object (2, 2a-2c) and the depth (z) and / or diameter (d) of the object (2, 2a-2c) are determined.

12. The system (50) according to claim 11, It is characterized in that The system (50) comprises a user interface, and the control and evaluation unit is configured to retrain the AI ​​model based on user input on the user interface, in particular corrections of the object position (x), depth (z) and / or diameter (d) determined by the system (50).

13. The system (50) according to claim 11 or 12, It is characterized in that The eddy current sensor (51) comprises a coil arrangement of a pair (52b) of connected, in particular circular coils surrounded by a single, in particular elliptical coil (52a), all coil axes being arranged in such a way that when measuring at the surface (55) using the eddy current sensor (51), all coil axes are perpendicular to the measuring surface (55).

14. The system (50) according to claim 13, It is characterized in that The control and evaluation unit is configured to: For the same sensing position (x), the coil pair sensor signals and the single coil sensor signals are measured and evaluated in parallel or in a double sequential sensing process, in particular during the forward (F) movement and the return (B) movement of the eddy current sensor (51), in order to determine the corresponding diameter (d) and / or depth (z) individually, and in the case of a sequential sensing process, At least one of the at least two determined diameter (d) and / or depth (z) values ​​is stored in a memory of the system for final fusion of the two diameter values ​​and / or depth values.

15. A computer program product having computer executable instructions for automatically performing the steps of the method according to claim 1, in particular on a system (50) according to claim 11.

Images