A method and system for residential in-wall plumbing detection and identification

Abstract

The application provides a kind of residential wall pipeline detection and identification method and system, belong to building engineering detection field.Its technical scheme is: including the following steps: S1, utilize infrared imaging and millimeter wave radar to the wall body to be measured is comprehensively detected, obtains original signal;S2, the original signal is pretreated and filtered, and the wall body structure characteristic data is extracted;S3, image processing and deep learning algorithm are applied to the wall body structure characteristic data and pipeline feature extraction;S4, the pipeline feature is classified and geometrically positioned, and the type and space position of pipeline are determined;S5, based on pipeline type and space position, generate visual image and output detection report.The beneficial effects of the application are: in the unified coordinate system, based on the synchronous acquisition of infrared and millimeter wave, space-time registration and adaptive parameter control, complete pixel-level consistency screening, fragment-level fusion classification and three-dimensional geometric positioning, and generate verifiable layer visualization and structured report.

Description

Technical Field

[0001] This invention relates to the field of building engineering inspection, and in particular to a method and system for detecting and identifying pipelines inside residential walls. Background Technology

[0002] In recent years, the demand for non-invasive inspection of in-wall pipes in building renovations and existing residential remodeling has grown rapidly. The industry has expanded from early magnetic and metal detectors and simple nail detectors to thermal infrared inspection and geological / millimeter-wave radar, with portable multi-sensor combination solutions emerging. Millimeter waves show potential in sensing media penetration and weak reflections, while infrared technology is being applied to temperature difference indication and operational status interpretation. Simultaneously, research focus has shifted from "equipment-oriented" to "data-algorithm-application," emphasizing cross-modal synchronous acquisition, spatiotemporal registration, feature fusion, 3D visualization, and traceable reporting.

[0003] However, existing solutions for residential walls are mostly single-sensor or loosely coupled integrations: thermal infrared is limited by surface emissivity and environmental drift, making it difficult to detect pipelines that are not in operation or are covered by insulation; geological / millimeter-wave radar is prone to multipath and strong coupling echoes under reinforced concrete and multi-layer plaster conditions, and manual thresholding methods are insufficient for distinguishing weak or adjacent targets; multimodal systems generally lack a unified time reference and pose calibration, making it difficult to align infrared images and radar data with the same grid and making it impossible to make pixel-level consistency judgments; the algorithm side often stays at the level of empirical rules and single-frame judgments, lacking joint feature modeling of "temperature-reflection-interface geometry", deep network probability output and fragment fusion under geometric constraints, making it difficult to provide traceable estimates of type confidence and three-dimensional position; the results are mostly screenshots or text descriptions, not bound to coordinate systems and confidence levels, and lack a structured reporting link. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for detecting and identifying pipelines within residential walls, based on synchronous acquisition, spatiotemporal registration, and adaptive parameter control of infrared and millimeter waves within a unified coordinate system. This system enables pixel-level consistency screening, fragment-level fusion classification, and three-dimensional geometric positioning, and generates verifiable layered visualization and structured reports.

[0005] This invention is achieved through the following measures:

[0006] A method for detecting and identifying pipelines within residential walls, characterized by comprising the following steps:

[0007] Infrared imaging and millimeter-wave radar are used to conduct a comprehensive detection of the wall under test and obtain the raw signals;

[0008] The original signal is preprocessed and filtered to extract the structural feature data of the wall.

[0009] Image processing and deep learning algorithms are used to extract pipeline features from wall structure feature data;

[0010] Classify and geometrically locate pipeline characteristics to determine the type and spatial location of pipelines;

[0011] Generates visual images based on pipeline type and spatial location and outputs inspection reports.

[0012] The invention also has the following specific features:

[0013] The wall under test is comprehensively detected using infrared imaging and millimeter-wave radar, and the raw signals obtained include:

[0014] Establish a unified time reference for infrared imaging and millimeter-wave radar and complete the geometric and range baseline calibration. Determine the installation distance from the wall, incident angle and scanning coverage area. Use synchronous triggering to jointly scan on the preset grid path, record the timestamp, pose and environmental parameters of each measurement point, and collect the infrared pixel matrix and millimeter-wave radar echo digital sequence respectively.

[0015] During the scanning process, the infrared range or integration time, millimeter-wave radar transmission power or time threshold and scanning step size are automatically adjusted based on the thresholds of dynamic range, signal-to-noise ratio and synchronization deviation.

[0016] The infrared pixel matrix and reference frame corresponding to time and space, the corresponding millimeter-wave radar echo sequence and reference echo, along with the station parameters are stored together, and the original signal is output.

[0017] The raw signal is preprocessed and filtered to extract wall structure feature data, including: spatiotemporal registration and coordinate mapping of the infrared pixel matrix and millimeter-wave radar echo digital sequence with a unified time reference and station parameters; background subtraction and range normalization are performed on the infrared pixel matrix according to the reference frame, and noise suppression and contrast normalization are performed to form temperature gradient and connected region candidates.

[0018] The digital sequence of millimeter-wave radar echoes is subjected to static background subtraction and time threshold clipping based on the reference echo, and bandpass or equivalent filtering and stacking averaging are performed to form range-reflection intensity mapping and continuity candidates.

[0019] The image processing and filtering parameters are adaptively adjusted and recorded according to the threshold strategy of dynamic range, signal-to-noise ratio and synchronization deviation, without changing the acquisition configuration; spatial correspondence and consistency checks are performed on infrared candidates and millimeter-wave candidates under unified coordinates, and the wall structure feature data including grid coordinates, surface and inner layer interface positions, reflection / temperature feature vectors and candidate linear target masks are output.

[0020] The application of image processing and deep learning algorithms to extract pipeline features from wall structure feature data includes:

[0021] Under a unified coordinate system, the wall structure feature data are fed into the image processing branch and the deep learning algorithm branch for parallel processing.

[0022] The image processing branch performs edge detection, thinning, and connected component analysis on candidate linear target masks, reflection feature vectors, and temperature feature vectors, and generates centerline point sets and direction vector sequences based on length continuity thresholds, curvature thresholds, and discontinuity tolerances.

[0023] The deep learning algorithm branch takes the reflection feature vector, temperature feature vector and the position of the surface and inner layer interface as multi-channel inputs, and uses a deep convolutional neural network to output pixel-level pipeline probability maps and fragment sets, and gives a confidence score and centerline estimate for each fragment.

[0024] Establish fusion and consistency check rules, perform spatial correspondence and threshold judgment on the results of image processing branch and deep learning algorithm branch in the grid coordinate corresponding area, retain the segments that simultaneously meet the pixel-level pipeline probability threshold and segment confidence threshold, perform secondary judgment on segments that only meet the single branch threshold based on neighborhood continuity score and geometric consistency with the surface and inner layer interface position, and perform deduplication on overlapping segments according to confidence priority and overlap threshold.

[0025] When any branch has insufficient candidates, low confidence of the centerline, or a synchronization deviation flag, it triggers a restrictive adjustment and recalculation of processing parameters such as edge strength threshold, line segment growth step size, and input normalization, without changing the acquisition configuration;

[0026] The fused centerline point set, orientation vector sequence, pixel-level pipeline probability, fragment confidence score and its index in grid coordinates, together with the initial depth estimate relative to the surface and inner interface position, constitute the pipeline features and are output.

[0027] The classification and geometric location of pipeline features, determining the type and spatial location of pipelines, includes:

[0028] The pipeline features, including the centerline point set, direction vector sequence, pixel-level pipeline probability, segment confidence score, index in grid coordinates, and initial depth estimate relative to the surface and inner interface, are fed into the pipeline classification module and the geometric localization module for parallel processing, respectively.

[0029] The pipeline classification module takes reflection feature vector, temperature feature vector, centerline point set and direction vector sequence as multi-channel input, and uses a multi-class deep learning classification model to output the category probability vectors of wire, water pipe and gas pipe. The pipeline type is selected according to the classification threshold. For segments below the classification threshold, the type is backdated based on the index continuity of adjacent segments in grid coordinates, the angle threshold of the direction vector sequence, pixel-level pipeline probability and segment confidence score.

[0030] The geometric positioning module maps the centerline point set from a unified coordinate system to the three-dimensional coordinate system of the actual coordinate system of the wall based on the station parameters, installation distance, incident angle, geometric and range baseline calibration, and time threshold. It uses the initial depth estimation to calculate the depth value of each centerline point and generates the starting three-dimensional coordinates, ending three-dimensional coordinates, and spatial direction vector of each segment.

[0031] After completing the classification and localization, a consistency check is performed. Overlapping segments in the same grid coordinate region are merged and deduplicated based on the segment confidence score and overlap threshold. When the geometric relationship between the orientation vector and the surface and inner layer interface position does not meet the preset consistency rules, segment reconstruction and index adjustment are triggered and the 3D coordinates and type labels are recalculated.

[0032] The output includes the pipeline type and spatial location of each pipeline, including the three-dimensional coordinates of the starting point, the three-dimensional coordinates of the ending point, and the spatial direction vector.

[0033] Generate a visual image and output an inspection report based on pipeline type and spatial location, including: establishing a visual coordinate frame in the actual coordinate system and grid coordinate system of the wall, loading the pipeline type, starting point 3D coordinates, ending point 3D coordinates, spatial direction vector, index in grid coordinates, surface and inner layer interface position and initial depth estimate of each pipeline, and generating a visual dataset;

[0034] Draw center lines in a unified coordinate system based on the visualization dataset and assign independent layers and display styles according to pipeline type to complete the mapping relationship between layers and pipeline numbers;

[0035] For overlapping segments appearing in the same grid coordinate area, perform overlap judgment and order arrangement, retain a single visualization path, and set numbered anchor points at both ends of the center line for report reference;

[0036] Before rendering, a consistency check is performed. If the coordinate field is missing, the coordinate is out of bounds, the pipeline type is not determined, or the geometric relationship between the spatial direction vector and the position of the surface and inner interface does not meet the preset rules, the data completion, confirmation mark or local geometry correction process is triggered and the visualization dataset is updated.

[0037] After rendering is completed, a visual image and an attached index are generated; the inspection report is arranged according to a preset template, reading the pipeline type, the three-dimensional coordinates of the starting point, the three-dimensional coordinates of the ending point, the spatial direction vector, the index in the grid coordinates, the acquisition timestamp and the station parameter number, forming a structured entry arranged by pipeline number, and embedding the corresponding attached index and visual image into the report;

[0038] During the arrangement process, a one-to-one correspondence check is performed between the entries and the visualization images. If there is a discrepancy, the process is returned to the visualization rendering flow to update before the arrangement is resumed. The visualization images are then exported and a detection report is output.

[0039] A system for detecting and identifying pipelines within residential walls, characterized in that it comprises:

[0040] Integrated Detection and Synchronous Acquisition Module: Utilizes infrared imaging and millimeter-wave radar to perform integrated detection of the wall under test and acquire raw signals;

[0041] Preprocessing and filtering / registration module: preprocesses and filters the raw signal to extract wall structure feature data;

[0042] Pipeline feature extraction module: Uses image processing and deep learning algorithms to extract pipeline features from wall structure feature data;

[0043] Classification and Geometric Location Module: Classifies and geometrically locates pipeline features to determine the type and spatial location of the pipeline;

[0044] Visualization and Inspection Report Module: Generates visual images and outputs inspection reports based on pipeline type and spatial location.

[0045] The beneficial effects of this invention are as follows: within a unified coordinate system, based on synchronous acquisition of infrared and millimeter waves, spatiotemporal registration and adaptive parameter control, pixel-level consistency screening, fragment-level fusion classification and three-dimensional geometric positioning are completed, and verifiable layered visualization and structured reports are generated. Attached Figure Description

[0046] Figure 1 This is an overall flowchart of a method for detecting and identifying pipelines inside residential walls, provided in an embodiment of the present invention. Detailed Implementation

[0047] To clearly illustrate the technical features of this solution, the following detailed implementation method will be used to describe the solution.

[0048] Example 1

[0049] See Figure 1 A method for detecting and identifying pipelines within residential walls, comprising the following steps:

[0050] Step S1: Utilize infrared imaging and millimeter-wave radar to perform comprehensive detection of the wall under test, acquiring the raw signals including:

[0051] Establish a unified time reference for infrared imaging and millimeter-wave radar and complete the geometric and range baseline calibration. Determine the installation distance from the wall, incident angle and scanning coverage area. Use synchronous triggering to jointly scan on the preset grid path, record the timestamp, pose and environmental parameters of each measurement point, and collect the infrared pixel matrix and millimeter-wave radar echo digital sequence respectively.

[0052] During the scanning process, the infrared range or integration time, millimeter-wave radar transmission power or time threshold and scanning step size are automatically adjusted based on the thresholds of dynamic range, signal-to-noise ratio and synchronization deviation.

[0053] The infrared pixel matrix and reference frame corresponding to time and space, the corresponding millimeter-wave radar echo sequence and reference echo, along with the station parameters are stored together, and the original signal is output.

[0054] Specifically, the infrared imaging equipment and the millimeter-wave radar use a unified time reference for trigger alignment, and complete the geometric and range baseline calibration before the detection begins;

[0055] Based on the calibration results, the installation distance from the wall, the incident angle, and the scanning coverage area of ​​the preset grid path are set; at each grid measuring point, a joint scan is performed through synchronous triggering, and the station parameters (time stamp, pose, environmental parameters) are recorded, and the infrared pixel matrix and millimeter-wave radar echo digital sequence are collected respectively.

[0056] To ensure that the two data streams maintain a usable dynamic range and measurable penetration depth under different wall materials and thermal field backgrounds, a "synchronous trigger + threshold-driven dynamic parameter tuning" mechanism is set up during the acquisition process:

[0057] When the online detection of dynamic range, signal-to-noise ratio and synchronization deviation reaches the preset threshold, the infrared range or integration time, millimeter-wave radar transmit power or time threshold are automatically recalculated, and the scanning step size is temporarily reduced under continuous low signal-to-noise ratio conditions; when the synchronization deviation recovers to within the threshold, the preset scanning step size is restored.

[0058] To facilitate implementation and ensure the formula's calculability, a distributed two-equation online control system with synchronous triggering closed-loop control is employed.

[0059] Infrared integral time adaptive control:

[0060] ,

[0061] Calculated by this formula Mapped to the infrared range and integration time interval allowed by the device, used to update the infrared acquisition configuration for the next frame;

[0062] Millimeter-wave radar power-time threshold linkage control:

[0063] ,

[0064] ,

[0065] This formula updates the millimeter-wave radar transmit power command at each measuring point in a coordinated manner. With time threshold This allows for coordinated scheduling of time delay windows and power between near-end strong echo suppression and deep echo visibility.

[0066] After each measurement point is collected, the infrared pixel matrix and reference frame corresponding to the current measurement point in time and space, the millimeter-wave radar echo digital sequence and reference echo, together with the station parameters, are bound and stored to form a spatiotemporal corresponding data set covering the preset grid path.

[0067] The final output of step S1 is the "original signal". The "original signal" consists of the above two measurement data, their reference data, and station parameters, and can be directly used in subsequent steps.

[0068] in, The infrared integration time normalization value is between 0 and 1, mapped by the device to the infrared range and integration time settings. A is the infrared dynamic range ratio, the ratio of the maximum to minimum pixel value in the current frame, with a value greater than or equal to 1. B is the infrared saturation ratio, the proportion of saturated pixels to the total number of pixels, with a value between 0 and 1. C is the temperature spatial gradient norm, the average gradient intensity of the temperature field, with a value greater than or equal to 0. D is the emissivity estimate, the equivalent infrared emissivity of the wall, with a value between 0 and 1. E is the environmental drift index, the difference between the temperature baseline of the reference frame and the current frame, with a real value. F is the sensor noise floor, the estimated infrared readout noise, with a value greater than or equal to 0. M is the millimeter-wave radar transmit power control value, with a value greater than 0. k is the power scale constant, calibrated by the device's rated power, with a value greater than 0. N is the target signal-to-noise ratio requirement, with a value greater than 0. The currently measured signal-to-noise ratio is greater than 0. To prevent the denominator from being zero, the robust term takes a value greater than 0. Q is the radar range estimate, which is the equivalent distance from the measurement point to the reflector inside the wall, and its value is greater than 0. The millimeter-wave incident angle is between 0 and π; R is the time threshold margin, an additional margin relative to the theoretical time delay window, with a value greater than or equal to 0; S is the synchronization deviation, a normalized amount of the time reference difference, with a real value; and W is the time threshold, the upper limit of the receiving window, with a value between 0 and π. between, The upper bound of the time threshold is set to a maximum receive delay supported by the device that is greater than 0. The equivalent propagation velocity of the wall is calculated from material parameters and is greater than 0. The delay bias term includes a fixed hardware delay value greater than or equal to 0.

[0069] Step S2: Preprocess and filter the original signal to extract wall structure feature data, including: performing spatiotemporal registration and coordinate mapping of the infrared pixel matrix and millimeter-wave radar echo digital sequence with a unified time reference and station parameters; performing background subtraction and range normalization on the infrared pixel matrix based on the reference frame, and performing noise suppression and contrast standardization to form temperature gradient and connected region candidates.

[0070] The digital sequence of millimeter-wave radar echoes is subjected to static background subtraction and time threshold clipping based on the reference echo, and bandpass or equivalent filtering and stacking averaging are performed to form range-reflection intensity mapping and continuity candidates.

[0071] The image processing and filtering parameters are adaptively adjusted and recorded according to the threshold strategy of dynamic range, signal-to-noise ratio and synchronization deviation, without changing the acquisition configuration; spatial correspondence and consistency checks are performed on infrared candidates and millimeter-wave candidates under unified coordinates, and the wall structure feature data including grid coordinates, surface and inner layer interface positions, reflection / temperature feature vectors and candidate linear target masks are output.

[0072] In step S2, the original signal output from step S1 is used as input. First, the spatiotemporal registration and coordinate mapping of the infrared pixel matrix and the millimeter-wave radar echo digital sequence are completed based on the unified time reference and station parameters, so that the infrared frames and range domain echoes in the same grid coordinate unit correspond one-to-one.

[0073] Subsequently, reference frame background subtraction and range normalization were performed on the infrared pixel matrix, and logarithmic-variance normalization and connected component analysis were performed in the local neighborhood to obtain enhanced temperature field and candidate connected regions.

[0074] For the digital sequence of millimeter-wave radar echoes, static background subtraction and time threshold clipping are performed based on the reference echo. Then, bandpass or wavelet equivalent filtering and multi-frame weighted superposition are applied to the truncated range domain data to form range-reflection intensity mapping and continuity candidates.

[0075] Then, based on the threshold strategy of dynamic range, signal-to-noise ratio and synchronization deviation, only adjust the image processing and filtering parameters in this step and record the adjustment, without changing the acquisition configuration;

[0076] Finally, spatial correspondence and consistency checks are performed on infrared and millimeter-wave candidates under a unified coordinate system, and wall structure feature data is output.

[0077] This step is implemented using a step-by-step formula;

[0078] Equation (1) Infrared local enhancement and connectivity candidates:

[0079] ,

[0080] in Used to calculate temperature gradients and extract candidate connected regions based on them.

[0081] Equation (2) Radar weighted superposition and soft threshold noise suppression:

[0082] ,

[0083] Equation (2) performs weighted superposition of multiple range domain echoes after bandpass or wavelet equivalent filtering and suppresses low-amplitude noise in a soft-threshold manner.

[0084] Equation (3) Cross-modal fusion mask:

[0085] ,

[0086] As a candidate linear target mask, it exhibits a high response only when the bimodal consistency is satisfied.

[0087] Equation (4) Interface depth estimation:

[0088] ,

[0089] ,

[0090] Equation (4) calculates the median depth of the peak set within each grid cell according to the percentile criterion, and obtains the location of the interface between the surface and the inner layer (if there are double peaks, the type will be distinguished in subsequent steps).

[0091] The wall structure feature data generated accordingly is a structured set: {grid coordinates, surface and inner layer interface position, temperature feature vector, reflection feature vector, candidate linear target mask}, which strictly corresponds to the spatiotemporal index of step S1 and is used for subsequent calls in step S3.

[0092] Value range description: A real number with near-zero mean and unit approximate variance stability is used for gradient and connectivity determination. Used for distance domain peak and continuity measurement. Used for grid-level candidate filtering Used for interface depth positioning.

[0093] Where: R is the pixel value of the infrared pixel after background subtraction from the reference frame and range normalization. Infrared horizontal pixel coordinates, Infrared vertical pixel coordinates, It is the logarithmic stability constant. For A set of local neighborhood indexes centered on the center. The mean of the neighborhood logarithm. The neighborhood logarithmic variance Let Variance be the stability constant. This is the result of local infrared enhancement. Radar range coordinates For overlaying frame indexes, To participate in the overlay of frames, For the first Frame weights For the first The range domain echo of the frame after static background subtraction, time threshold clipping, and bandpass or wavelet equivalent filtering. This is the soft threshold steepness coefficient. For soft threshold position parameters, For the equivalent reflection intensity of multiple frames, For grid horizontal indexing, For grid vertical indexing, Mapping to the grid Distance continuity index These are the temperature gradient weighting coefficients. For continuous weighting coefficients, To achieve the fusion bias, To fuse candidate masks, For grid The corresponding distance sample set, The threshold is the percentage of the peak value. For interface depth estimation, This represents the maximum usable depth.

[0094] Step S3, applying image processing and deep learning algorithms to extract pipeline features from the wall structure feature data, includes:

[0095] Under a unified coordinate system, the wall structure feature data are fed into the image processing branch and the deep learning algorithm branch for parallel processing.

[0096] The image processing branch performs edge detection, thinning, and connected component analysis on candidate linear target masks, reflection feature vectors, and temperature feature vectors, and generates centerline point sets and direction vector sequences based on length continuity thresholds, curvature thresholds, and discontinuity tolerances.

[0097] The deep learning algorithm branch takes the reflection feature vector, temperature feature vector and the position of the surface and inner layer interface as multi-channel inputs, and uses a deep convolutional neural network to output pixel-level pipeline probability maps and fragment sets, and gives a confidence score and centerline estimate for each fragment.

[0098] Establish fusion and consistency check rules, perform spatial correspondence and threshold judgment on the results of image processing branch and deep learning algorithm branch in the grid coordinate corresponding area, retain the segments that simultaneously meet the pixel-level pipeline probability threshold and segment confidence threshold, perform secondary judgment on segments that only meet the single branch threshold based on neighborhood continuity score and geometric consistency with the surface and inner layer interface position, and perform deduplication on overlapping segments according to confidence priority and overlap threshold.

[0099] When any branch has insufficient candidates, low confidence of the centerline, or a synchronization deviation flag, it triggers a restrictive adjustment and recalculation of processing parameters such as edge strength threshold, line segment growth step size, and input normalization, without changing the acquisition configuration;

[0100] The fused centerline point set, orientation vector sequence, pixel-level pipeline probability, fragment confidence score and its index in grid coordinates, together with the initial depth estimate relative to the surface and inner interface position, constitute the pipeline features and are output.

[0101] In step S3, the wall structure feature data output in step S2 is used as input, and the image processing branch and the deep learning algorithm branch are started in parallel under a unified coordinate system.

[0102] The image processing branch performs edge detection, skeleton thinning, and connected component analysis only within the grid defined by the candidate linear target mask, and generates a centerline point set and a direction vector sequence under the constraints of length continuity threshold, curvature threshold, and discontinuity tolerance.

[0103] The deep learning algorithm branch encodes the reflection feature vector, temperature feature vector, and surface and inner layer interface position into a multi-channel tensor input deep convolutional network, and outputs a pixel-level pipeline probability map, a fragment set, an initial confidence value for each fragment, and a centerline estimate.

[0104] In the fusion phase, spatial correspondence and threshold judgment are performed in the corresponding areas of the grid coordinates: First, the basic confidence score at the fragment level is calculated to unify and merge the pixel probabilities, distance continuity, and temperature gradients of the two branches; then, geometric smoothing and depth consistency correction are introduced to suppress curvature abrupt changes and cross-interface inconsistencies; for fragments that only meet the single-branch threshold, a secondary judgment is made based on neighborhood continuity scoring and geometric consistency with the surface and inner layer interface positions; overlapping fragments are prioritized according to confidence score and deduplicated based on the overlap threshold; when any branch has insufficient candidates, low centerline confidence, or synchronization deviation markers, only the edge intensity threshold, line segment growth step size, and input normalization parameters are adjusted and recalculated within this step, without changing the acquisition configuration. The final output is a "pipeline feature" consisting of the centerline point set, direction vector sequence, pixel-level pipeline probability, fragment fusion confidence score, grid coordinate index, and initial depth estimate relative to the surface and inner layer interface positions.

[0105] To ensure that the fusion screening is calculable and traceable, a step-by-step formula is adopted:

[0106] Equation (1) Fragment-level basic confidence aggregation

[0107] ,

[0108] Equation (2) Geometric and depth correction and acceptance criteria

[0109] ,

[0110] ,

[0111] Accordingly, the instruction is accepted. The fragments are preserved and used in centerline splicing and direction vector generation to form the final "pipeline features", which are then used in step S4.

[0112] Value range description: .

[0113] Where: s is the fragment index set element, representing a candidate line segment. The number of raster pixels contained in this segment. For grid horizontal and vertical indexes, These are the pixel-level pipeline probabilities output by the deep learning branch. As a measure of distance continuity mapped to a raster, The temperature gradient intensity mapped to the grid. These are the pixel probability channel coefficients. For distance continuity channel coefficients, This represents the temperature gradient channel coefficient. Let be the gradient stability constant. This represents the basic confidence level at the fragment level. It is the hyperbolic tangent function. It is an exponential function. Pi is a constant. The average rotation angle of adjacent difference vectors along the centerline of a segment is obtained by averaging the angles between the difference vectors of the centerline sequence. This represents the minimum depth gap within the corresponding raster relative to the surface and inner layer interface position of the fragment. The geometric smoothing gain coefficient, This represents the depth consistency gain coefficient. For the fusion bias constant, For fragment fusion confidence, To accept the threshold, Accept indicator function for fragment The output is a binary result.

[0114] Step S4: Classify and geometrically locate pipeline features to determine the type and spatial location of the pipeline, including:

[0115] The pipeline features, including the centerline point set, direction vector sequence, pixel-level pipeline probability, segment confidence score, index in grid coordinates, and initial depth estimate relative to the surface and inner interface, are fed into the pipeline classification module and the geometric localization module for parallel processing, respectively.

[0116] The pipeline classification module takes reflection feature vector, temperature feature vector, centerline point set and direction vector sequence as multi-channel input, and uses a multi-class deep learning classification model to output the category probability vectors of wire, water pipe and gas pipe. The pipeline type is selected according to the classification threshold. For segments below the classification threshold, the type is backdated based on the index continuity of adjacent segments in grid coordinates, the angle threshold of the direction vector sequence, pixel-level pipeline probability and segment confidence score.

[0117] The geometric positioning module maps the centerline point set from a unified coordinate system to the three-dimensional coordinate system of the actual coordinate system of the wall based on the station parameters, installation distance, incident angle, geometric and range baseline calibration, and time threshold. It uses the initial depth estimation to calculate the depth value of each centerline point and generates the starting three-dimensional coordinates, ending three-dimensional coordinates, and spatial direction vector of each segment.

[0118] After completing the classification and localization, a consistency check is performed. Overlapping segments in the same grid coordinate region are merged and deduplicated based on the segment confidence score and overlap threshold. When the geometric relationship between the orientation vector and the surface and inner layer interface position does not meet the preset consistency rules, segment reconstruction and index adjustment are triggered and the 3D coordinates and type labels are recalculated.

[0119] The output includes the pipeline type and spatial location of each pipeline, including the three-dimensional coordinates of the starting point, the three-dimensional coordinates of the ending point, and the spatial direction vector.

[0120] In step S4, the pipeline features output in step S3 are used as input, and the pipeline classification module and the geometric positioning module are started in parallel under a unified coordinate system.

[0121] The inputs include: centerline point set, direction vector sequence, pixel-level pipeline probability, fragment confidence score, index in grid coordinates, and initial depth estimate relative to the surface and inner interface position.

[0122] The classification module constructs a multi-channel description (reflection feature vector, temperature feature vector, and centerline and direction statistics) for each candidate segment, and the deep learning classifier outputs the category probability vectors of wire, water pipe, and gas pipe.

[0123] When the maximum category probability is lower than the classification threshold, a backoff decision is triggered, which is based on a comprehensive decision made on the continuity of the index of adjacent segments in the grid coordinates, the threshold of the angle between the direction vector sequences, the pixel-level pipeline probability, and the segment confidence score.

[0124] The geometric positioning module maps the centerline endpoints from a unified coordinate system to the actual coordinate system of the wall based on the station parameters, installation distance, incident angle, geometric and range baseline calibration, and time threshold. It also generates the starting three-dimensional coordinates, ending three-dimensional coordinates, and spatial orientation vector of each segment by combining the initial depth estimation. After classification and positioning are completed, a consistency check is performed within the same grid coordinate area. Overlapping segments are merged and deduplicated based on the segment confidence score and overlap threshold.

[0125] When the geometric relationship between the directional vector and the positions of the surface and inner interface does not meet the preset consistency rules, fragment reconstruction and index adjustment are triggered, and the 3D coordinates and type annotations are recalculated. This step focuses on the workflow; formulas are only used as auxiliary tools and for record-keeping.

[0126] Equation (1) — Fragment category posterior fusion

[0127] ,

[0128] Equation (1) will be used in the classifier evidence Based on this, a grid continuity metric is introduced. Consistency measurement with the trend The temperature-based nonlinear fusion is used as the three-class posterior probability, which facilitates docking with the classification threshold and triggering backoff.

[0129] Equation (2) – Endpoint 3D Mapping and Depth Injection

[0130] ,

[0131] ,

[0132] Equation (2) converts the endpoint grid coordinates Calibrated homography matrix Project onto the wall coordinate plane and use an initial depth estimate. Interface Difference Metrics Generate along the wall normal Continuous depth displacement enables stable calculation of the three-dimensional coordinates of the starting and ending points.

[0133] Equation (3) – Direction and Fusion Scoring and Deduplication Criteria

[0134] ,

[0135] ,

[0136] Equation (3) calculates the unit direction vector. And provide a fusion score. (The product of the maximum class posterior and the segment confidence passed in step S3) is used to filter and retain segments within the overlapping set based on the overlap threshold. The largest fragments are merged after deduplication.

[0137] Based on this, the final output pipeline type and spatial location are: the type is determined by... Determined; spatial location determined by (starting point three-dimensional coordinates) (End point three-dimensional coordinates) and The vector is composed of spatial orientation vectors and maintains a one-to-one correspondence with the input grid coordinate indices;

[0138] Value range description: and Used for type determination; The endpoints are in three-dimensional coordinates; For a unit vector to satisfy ; Used for priority sorting of overlapping segments.

[0139] in: The segment index is used to identify a single candidate pipeline segment, and t is the set of category index values ​​(E represents the electrical wire category, W represents the water pipe category, and G represents the gas pipe category). For fragments Regarding the posterior probability of category t, For the classification evidence gain coefficient, For continuous gain coefficient, To achieve a consistent gain coefficient, For deep learning classifiers to classify fragments The amount of evidence in category t, For fragments Measure of index continuity in grid coordinates For fragments A measure of consistency in trends; The homography matrix is ​​obtained from the geometric and range baseline calibration. For fragments The horizontal coordinates of the starting point grid. For fragments The vertical coordinate of the starting point grid. For fragments The horizontal coordinates of the grid at the endpoint. For fragments The vertical coordinate of the grid at the endpoint, The depth scale constant is obtained from the station parameters and installation distance calibration. The depth bias constant is used to compensate for the fixed error of the system. For fragments The initial depth estimate relative to the surface-to-inner-layer interface location, For fragments The interface difference index is mapped to the real number domain. Let be the unit vector normal to the outside of the wall. For fragments The three-dimensional coordinate vector of the starting point, For fragments The three-dimensional coordinate vector of the endpoint For fragments The unit direction vector, The segment fusion confidence score passed in step S3, For fragments The fusion score is used for merging and deduplicating overlapping segments.

[0140] Step S5: Generate a visual image and output an inspection report based on pipeline type and spatial location. This includes: establishing a visual coordinate frame in the actual coordinate system and grid coordinate system of the wall, loading the pipeline type, starting point 3D coordinates, ending point 3D coordinates, spatial direction vector, index in grid coordinates, surface and inner layer interface position, and initial depth estimate for each pipeline, and generating a visual dataset.

[0141] Draw center lines in a unified coordinate system based on the visualization dataset and assign independent layers and display styles according to pipeline type to complete the mapping relationship between layers and pipeline numbers;

[0142] For overlapping segments appearing in the same grid coordinate area, perform overlap judgment and order arrangement, retain a single visualization path, and set numbered anchor points at both ends of the center line for report reference;

[0143] Before rendering, a consistency check is performed. If the coordinate field is missing, the coordinate is out of bounds, the pipeline type is not determined, or the geometric relationship between the spatial direction vector and the position of the surface and inner interface does not meet the preset rules, the data completion, confirmation mark or local geometry correction process is triggered and the visualization dataset is updated.

[0144] After rendering is completed, a visual image and an attached index are generated; the inspection report is arranged according to a preset template, reading the pipeline type, the three-dimensional coordinates of the starting point, the three-dimensional coordinates of the ending point, the spatial direction vector, the index in the grid coordinates, the acquisition timestamp and the station parameter number, forming a structured entry arranged by pipeline number, and embedding the corresponding attached index and visual image into the report;

[0145] During the arrangement process, a one-to-one correspondence check is performed between the entries and the visualization images. If there is a discrepancy, the process is returned to the visualization rendering flow to update before the arrangement is resumed. The visualization images are then exported and a detection report is output.

[0146] In step S5, using the pipeline type and spatial location output from step S4 as input, a visualization coordinate framework for rendering and report arrangement is established, and a visualization dataset is generated;

[0147] The process is as follows: First, a unified mapping is constructed between the actual coordinate system of the wall and the grid coordinate system. Then, the pipeline type, starting three-dimensional coordinates, ending three-dimensional coordinates, spatial direction vector, index in the grid coordinates, surface and inner layer interface position, and initial depth estimate of each pipeline are loaded. Finally, the completeness of the fields and the numerical boundaries are checked.

[0148] Then, independent layers and display styles are assigned according to pipeline type, and a "layer-pipeline number" mapping is established;

[0149] For overlapping segments in the same grid area, perform overlap judgment and drawing order arrangement, retain only a single visual path and set numbered anchor points at both ends of the center line;

[0150] Before rendering, a consistency check is performed. If a missing coordinate field, out-of-bounds coordinate, undetermined pipeline type, or geometric relationship between the spatial orientation vector and the position of the surface and inner interface does not meet the preset rules, data completion, pending confirmation marking, or local geometric correction are triggered and the visualization dataset is updated. After rendering is completed, a visualization image and attached index are generated.

[0151] The test report is compiled according to a preset template, which reads the pipeline type, the three-dimensional coordinates of the starting point and the three-dimensional coordinates of the ending point, the spatial direction vector, the index in the grid coordinates, the acquisition timestamp and the station parameter number, forming a structured entry arranged by pipeline number, and embedding the corresponding attached map index and visualization image into the report;

[0152] During the arrangement process, a one-to-one correspondence check is performed between the entries and the visualization images. If there is a discrepancy, the process is returned to the visualization rendering flow to update before the arrangement is resumed.

[0153] Export and generate a visual image and output a detection report;

[0154] Equation (1) — Coordinate unification and lightweight correction (for integrated visualization of coordinates)

[0155] ,

[0156] Equation (1) introduces a small-amplitude secondary correction based on the rigid transformation to compensate for the non-ideal flatness of the wall or the cumulative measurement error, and obtains the three-dimensional coordinates for rendering. ;

[0157] Equation (2) — Depth trade-off and visibility (used for anchor point and occlusion relationship)

[0158] ,

[0159] ,

[0160] Equation (2) uses the angle between the direction and the normal. Adjusting the surface / inner layer depth contribution and mapping it to a visibility metric via Sigmoid. Used for anchor point priority and layer occlusion determination;

[0161] Equation (3) — Direction-sensitive overlap score (used for deduplication and drawing order)

[0162] ,

[0163] ,

[0164] ,

[0165] ,

[0166] ,

[0167] Equation (3) couples the line segment projection overlap ratio with the directional consistency into a score. A higher value is only generated when the paths are highly parallel and overlapping. Based on this, the highest-scoring paths are filtered and retained using an overlap threshold to determine the drawing order.

[0168] The coordinate unification is completed by equation (1), the anchor point and occlusion constraint are completed by equation (2), and the overlap deduplication and order arrangement are completed by equation (3). With the cooperation of consistency verification and templated arrangement, a one-to-one correspondence result of "visualized image + attached index + structured item" is formed. Finally, the visualized image is generated and the detection report is output.

[0169] in: The grid's horizontal coordinates are real numbers and measurable. For grid vertical coordinates to be real numbers and measurable. The grid normal coordinates are real numbers and measurable. The rotation matrix from the grid system to the actual wall system is a third-order real matrix. Let the translation vector from the grid system to the actual wall system be a three-dimensional real vector. The horizontal quadratic correction coefficient is a real number. The longitudinal quadratic correction coefficient is a real number. The normal quadratic correction coefficient is a real number. The 3D coordinates used for rendering are 3D real vectors. The angle between the pipeline route and the external normal of the wall is between 0 and... between, The depth of the surface interface is a non-negative real number. The depth of the inner interface is a non-negative real number. For visibility steepness coefficients to be positive real numbers, For visibility reference depth, a non-negative real number is used. The weighted depth estimate is a non-negative real number. For visibility metrics, the values ​​are between 0 and 1. The three-dimensional coordinates of the starting point of the first segment's centerline are three-dimensional real vectors. The three-dimensional coordinates of the endpoint of the first segment's centerline are given by a three-dimensional real vector. The first segment has a length of a positive real number. The first segment's unit direction vector is a three-dimensional real vector with a norm of 1. The three-dimensional coordinates of the starting point of the centerline of the second segment are three-dimensional real vectors. The three-dimensional coordinates of the endpoint of the second segment's centerline are given by a three-dimensional real vector. The second segment has a length of a positive real number. The second segment's unit direction vector is a three-dimensional real vector with a norm of 1. The projection distance of the starting point of the second segment in the direction of the first segment is a real number. The projection distance of the endpoint of the second segment onto the direction of the first segment is a real number. The overlap length of the two segments projected onto the first segment is a non-negative real number. The direction-sensitive overlap score is set between 0 and 1 and is used for deduplication and drawing order determination.

[0170] Example 2

[0171] A system for detecting and identifying pipelines within residential walls, characterized in that it comprises:

[0172] Integrated Detection and Synchronous Acquisition Module: Utilizes infrared imaging and millimeter-wave radar to perform integrated detection of the wall under test and acquire raw signals;

[0173] Preprocessing and filtering / registration module: preprocesses and filters the raw signal to extract wall structure feature data;

[0174] Pipeline feature extraction module: Uses image processing and deep learning algorithms to extract pipeline features from wall structure feature data;

[0175] Classification and Geometric Location Module: Classifies and geometrically locates pipeline features to determine the type and spatial location of the pipeline;

[0176] Visualization and Inspection Report Module: Generates visual images and outputs inspection reports based on pipeline type and spatial location.

[0177] The technical features of this invention not described can be implemented by or using existing technology, and will not be repeated here. Of course, the above description is not a limitation of this invention, and this invention is not limited to the examples above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of this invention should also be within the protection scope of this invention.

Claims

1. A method for detecting and identifying pipelines within residential walls, characterized in that, Includes the following steps: Infrared imaging and millimeter-wave radar are used to conduct a comprehensive detection of the wall under test and obtain the raw signals; The original signal is preprocessed and filtered to extract the structural feature data of the wall. Image processing and deep learning algorithms are used to extract pipeline features from wall structure feature data; Classify and geometrically locate pipeline characteristics to determine the type and spatial location of pipelines; Generate visual images and output inspection reports based on pipeline type and spatial location; The application of image processing and deep learning algorithms to extract pipeline features from wall structure feature data includes: Under a unified coordinate system, the wall structure feature data are fed into the image processing branch and the deep learning algorithm branch for parallel processing. The image processing branch performs edge detection, thinning, and connected component analysis on candidate linear target masks, reflection feature vectors, and temperature feature vectors, and generates centerline point sets and direction vector sequences based on length continuity thresholds, curvature thresholds, and discontinuity tolerances. The deep learning algorithm branch takes the reflection feature vector, temperature feature vector and the position of the surface and inner layer interface as multi-channel inputs, and uses a deep convolutional neural network to output pixel-level pipeline probability maps and fragment sets, and gives a confidence score and centerline estimate for each fragment. Establish fusion and consistency check rules, perform spatial correspondence and threshold judgment on the results of image processing branch and deep learning algorithm branch in the grid coordinate corresponding area, retain the segments that simultaneously meet the pixel-level pipeline probability threshold and segment confidence threshold, perform secondary judgment on segments that only meet the single branch threshold based on neighborhood continuity score and geometric consistency with the surface and inner layer interface position, and perform deduplication on overlapping segments according to confidence priority and overlap threshold. When any branch has insufficient candidates, low confidence of the centerline, or a synchronization deviation flag, the restrictive adjustment and recalculation of the edge strength threshold, line segment growth step size, and input normalization processing parameters are triggered without changing the acquisition configuration. The fused centerline point set, orientation vector sequence, pixel-level pipeline probability, fragment confidence score and its index in grid coordinates, together with the initial depth estimate relative to the surface and inner interface position, constitute the pipeline features and are output.

2. The method for detecting and identifying pipelines within residential walls according to claim 1, characterized in that, The wall under test is comprehensively detected using infrared imaging and millimeter-wave radar, and the raw signals obtained include: Establish a unified time reference for infrared imaging and millimeter-wave radar and complete the geometric and range baseline calibration. Determine the installation distance from the wall, incident angle and scanning coverage area. Use synchronous triggering to jointly scan on the preset grid path, record the timestamp, pose and environmental parameters of each measurement point, and collect the infrared pixel matrix and millimeter-wave radar echo digital sequence respectively. During the scanning process, the infrared range or integration time, millimeter-wave radar transmission power or time threshold and scanning step size are automatically adjusted based on the thresholds of dynamic range, signal-to-noise ratio and synchronization deviation. The infrared pixel matrix and reference frame corresponding to time and space, the corresponding millimeter-wave radar echo sequence and reference echo, along with the station parameters are stored together, and the original signal is output.

3. The method for detecting and identifying pipelines within residential walls according to claim 2, characterized in that, The raw signal is preprocessed and filtered to extract wall structure feature data, including: spatiotemporal registration and coordinate mapping of the infrared pixel matrix and millimeter-wave radar echo digital sequence with a unified time reference and station parameters; background subtraction and range normalization are performed on the infrared pixel matrix according to the reference frame, and noise suppression and contrast normalization are performed to form temperature gradient and connected region candidates. The digital sequence of millimeter-wave radar echoes is subjected to static background subtraction and time threshold clipping based on the reference echo, and bandpass or equivalent filtering and stacking averaging are performed to form range-reflection intensity mapping and continuity candidates. The image processing and filtering parameters are adaptively adjusted and recorded according to the threshold strategy of dynamic range, signal-to-noise ratio and synchronization deviation, without changing the acquisition configuration; spatial correspondence and consistency checks are performed on infrared candidates and millimeter-wave candidates under unified coordinates, and the wall structure feature data including grid coordinates, surface and inner layer interface positions, reflection / temperature feature vectors and candidate linear target masks are output.

4. The method for detecting and identifying pipelines within residential walls according to claim 3, characterized in that, The classification and geometric location of pipeline features, determining the type and spatial location of pipelines, includes: The pipeline features, including the centerline point set, direction vector sequence, pixel-level pipeline probability, segment confidence score, index in grid coordinates, and initial depth estimate relative to the surface and inner interface, are fed into the pipeline classification module and the geometric localization module for parallel processing, respectively. The pipeline classification module takes reflection feature vector, temperature feature vector, centerline point set and direction vector sequence as multi-channel input, and uses a multi-class deep learning classification model to output the category probability vectors of wire, water pipe and gas pipe. The pipeline type is selected according to the classification threshold. For segments below the classification threshold, the type is backdated based on the index continuity of adjacent segments in grid coordinates, the angle threshold of the direction vector sequence, pixel-level pipeline probability and segment confidence score. The geometric positioning module maps the centerline point set from a unified coordinate system to the three-dimensional coordinate system of the actual coordinate system of the wall based on the station parameters, installation distance, incident angle, geometric and range baseline calibration, and time threshold. It uses the initial depth estimation to calculate the depth value of each centerline point and generates the starting three-dimensional coordinates, ending three-dimensional coordinates, and spatial direction vector of each segment. After completing the classification and localization, a consistency check is performed. Overlapping segments in the same grid coordinate region are merged and deduplicated based on the segment confidence score and overlap threshold. When the geometric relationship between the orientation vector and the surface and inner layer interface position does not meet the preset consistency rules, segment reconstruction and index adjustment are triggered and the 3D coordinates and type labels are recalculated. The output includes the pipeline type and spatial location of each pipeline, including the three-dimensional coordinates of the starting point, the three-dimensional coordinates of the ending point, and the spatial direction vector.

5. The method for detecting and identifying pipelines within residential walls according to claim 4, characterized in that, Generate a visual image and output an inspection report based on pipeline type and spatial location, including: establishing a visual coordinate frame in the actual coordinate system and grid coordinate system of the wall, loading the pipeline type, starting point 3D coordinates, ending point 3D coordinates, spatial direction vector, index in grid coordinates, surface and inner layer interface position and initial depth estimate of each pipeline, and generating a visual dataset; Draw center lines in a unified coordinate system based on the visualization dataset and assign independent layers and display styles according to pipeline type to complete the mapping relationship between layers and pipeline numbers; For overlapping segments appearing in the same grid coordinate area, perform overlap judgment and order arrangement, retain a single visualization path, and set numbered anchor points at both ends of the center line for report reference; Before rendering, a consistency check is performed. If the coordinate field is missing, the coordinate is out of bounds, the pipeline type is not determined, or the geometric relationship between the spatial direction vector and the position of the surface and inner interface does not meet the preset rules, the data completion, confirmation mark or local geometry correction process is triggered and the visualization dataset is updated. After rendering is completed, a visual image and an attached index are generated; the inspection report is arranged according to a preset template, reading the pipeline type, the three-dimensional coordinates of the starting point, the three-dimensional coordinates of the ending point, the spatial direction vector, the index in the grid coordinates, the acquisition timestamp and the station parameter number, forming a structured entry arranged by pipeline number, and embedding the corresponding attached index and visual image into the report; During the arrangement process, a one-to-one correspondence check is performed between the entries and the visualization images. If there is a discrepancy, the process is returned to the visualization rendering flow to update before the arrangement is resumed. The visualization images are then exported and a detection report is output.

6. A system employing the method for detecting and identifying pipelines within residential walls as described in any one of claims 1-5, characterized in that, include: Integrated Detection and Synchronous Acquisition Module: Utilizes infrared imaging and millimeter-wave radar to perform integrated detection of the wall under test and acquire raw signals; Preprocessing and filtering / registration module: preprocesses and filters the raw signal to extract wall structure feature data; Pipeline feature extraction module: Uses image processing and deep learning algorithms to extract pipeline features from wall structure feature data; Classification and Geometric Location Module: Classifies and geometrically locates pipeline features to determine the type and spatial location of the pipeline; Visualization and Inspection Report Module: Generates visual images and outputs inspection reports based on pipeline type and spatial location.

Images