Method and system for recognizing water leakage of elevator shaft wall based on three-dimensional point cloud and image
By combining the joint analysis of 3D point clouds and images with deep learning models, and considering the material of the shaft wall, the problems of missed and false detections in elevator shaft wall leakage detection were solved, achieving comprehensive and accurate identification of the shaft wall and generation of detailed reports.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-27
- Publication Date
- 2026-03-13
Smart Images
Figure CN121661448A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, specifically to a method and system for identifying water leakage in elevator shaft walls based on three-dimensional point clouds and images. Background Technology
[0002] As an indispensable vertical transportation tool in modern buildings, the safe operation of elevators is of paramount importance. The elevator shaft, as the enclosed space where the elevator car and counterweight operate, is directly affected by its structural health. Water leakage may occur in the shaft walls due to material aging, seal failure, or external moisture penetration. This not only corrodes internal metal components such as guide rails and cables, leading to equipment corrosion and malfunctions, but also poses a risk of icing in extremely cold regions, seriously threatening elevator operational safety.
[0003] Currently, methods for detecting building water leakage mainly include manual visual inspection, automatic recognition technology based on two-dimensional images, and morphological analysis technology based on three-dimensional point clouds. Manual visual inspection relies on the experience of inspectors, but elevator shafts are vertical, enclosed spaces with significant physical obstructions, limiting the inspector's field of vision, resulting in low detection efficiency and safety risks. Two-dimensional image-based recognition technology is susceptible to changes in lighting and surface texture interference, making it difficult to distinguish water stains from shadows or other contaminants, especially when the leakage area is partially or completely obscured by equipment within the shaft, rendering this method essentially ineffective. While three-dimensional point cloud technology can reconstruct the structural surface, point cloud data lacks color and texture information, making it difficult to accurately identify the unique color and saturation variations of leaking water. Furthermore, point clouds themselves suffer from data gaps in obscured areas. Existing technologies are insufficient in terms of versatility, accuracy, and comprehensiveness in complex shaft environments. Summary of the Invention
[0004] This invention addresses the technical problems of existing technologies lacking a deep fusion mechanism of three-dimensional and two-dimensional information, which leads to missed detections, false detections, and insufficient versatility in elevator shaft wall leakage identification. It provides a method and system for elevator shaft wall leakage identification based on three-dimensional point cloud and image.
[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: In a first aspect, the present invention provides a method for identifying water leakage in elevator shaft walls based on three-dimensional point clouds and images, including: The point cloud image and captured image of the elevator shaft wall area to be identified are obtained, and joint occlusion analysis is performed to obtain multiple occluded areas and multiple non-occluded areas. Leakage is identified in multiple unobstructed areas to obtain multiple leakage areas, and feature matching is performed on multiple obstructed areas and multiple leakage areas to obtain multiple suspected leakage areas; Based on the well wall material, multiple suspected leakage areas are verified for leakage, and multiple leakage verification results are obtained. The leakage verification results include leakage probability and leakage pattern. By integrating multiple leakage verification results and multiple leakage areas, an elevator shaft wall leakage identification report is obtained and output.
[0006] Secondly, the present invention provides an elevator shaft wall leakage identification system based on three-dimensional point cloud and image, comprising: The data acquisition and joint analysis module is used to acquire point cloud images and captured images of the elevator shaft wall area to be identified, and to perform joint occlusion analysis to acquire multiple occluded areas and multiple unoccluded areas. The leakage identification and feature matching module is used to identify leakage in multiple unobstructed areas, obtain multiple leakage areas, and perform feature matching on multiple obstructed areas and multiple leakage areas to obtain multiple suspected leakage areas. The leakage verification module is used to verify the leakage of multiple suspected leakage areas in combination with the well wall material, and obtain multiple leakage verification results, wherein the leakage verification results include leakage probability and leakage mode. The report generation and output module is used to integrate multiple leakage verification results and multiple leakage areas to obtain an elevator shaft wall leakage identification report and output it.
[0007] The beneficial effects of this invention are: Compared to existing technologies, this invention firstly uses joint occlusion analysis of 3D point clouds and 2D images to accurately delineate occluded and unoccluded areas, effectively overcoming the obstruction of the detection field by the complex structure within the wellbore and reducing the risk of missed leaks due to data gaps. Secondly, while directly identifying leaks in unoccluded areas, it innovatively performs cross-modal matching between the identified leak area features and occluded areas, thereby inferring the suspected leak areas behind the occluded areas and achieving comprehensive detection of both visible and invisible areas. Thirdly, the invention introduces the wellbore wall material as a key factor in the verification stage, constructing a leak verifier adapted to different material properties through transfer learning technology. This effectively distinguishes between real leaks and interference phenomena such as condensation and stains, significantly improving the accuracy of the identification results and their versatility across different application scenarios. Finally, by structurally integrating identification and verification results from different sources and with varying degrees of credibility, a detailed and reliable leak detection report is generated, providing an intuitive and comprehensive decision-making basis for subsequent precise maintenance. Attached Figure Description
[0008] Figure 1 This is a flowchart illustrating the elevator shaft wall leakage identification method based on three-dimensional point cloud and image provided by the present invention. Figure 2This is a schematic diagram of the elevator shaft wall leakage identification system based on three-dimensional point cloud and image provided by the present invention.
[0009] In the attached diagram, the components represented by each number are as follows: The module includes: data acquisition and joint analysis module 11, leakage identification and feature matching module 12, leakage verification module 13, and report generation and output module 14. Detailed Implementation
[0010] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0011] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.
[0012] In the description of this invention, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed herein.
[0013] Example 1, as Figure 1 As shown, this embodiment of the invention provides a method for identifying water leakage in elevator shaft walls based on three-dimensional point clouds and images, including: S10: Acquire point cloud images and captured images of the elevator shaft wall area to be identified, and perform joint occlusion analysis to obtain multiple occluded areas and multiple non-occluded areas; The elevator shaft wall area to be identified refers to the vertical wall surface inside the elevator shaft that needs to be tested for water leakage. This includes continuous or segmented walls, such as key areas including concrete joints, the perimeter of embedded parts, the perimeter of landing doors, and the fixing points of guide rail supports. This area is a spatial range covered by both 3D point cloud data and 2D optical image data. By identifying the abnormal humidity characteristics and distribution patterns caused by water leakage within this area, the source of leakage can be accurately located and its impact range assessed. This provides a precise basis for decision-making regarding the structural safety maintenance and targeted waterproofing of the elevator shaft.
[0014] Specifically, point cloud images and captured images of the elevator shaft wall region to be identified are obtained, and joint occlusion analysis is performed to obtain multiple occlusion regions, including: Acquire point cloud images and captured images of the elevator shaft wall area to be identified, and perform point cloud image feature calculation to obtain point cloud void ratio and point cloud edge features; Obtain the image edge features of multiple captured images; The point cloud image and multiple captured images are registered to obtain the matching degree between the edge features of the point cloud and the edge features of the image. Regions with a point cloud hole rate greater than a preset hole threshold and a projection matching degree greater than a preset matching threshold are marked as potential occlusion areas. The point cloud edge features and image edge features of the potential occlusion area are jointly determined to filter and obtain the occlusion area.
[0015] First, point cloud images and captured images of the elevator shaft wall area to be identified are acquired. Point cloud images are acquired using devices such as laser scanners, recording the three-dimensional geometric information of the shaft wall surface in the form of point sets; captured images are acquired using optical cameras, recording the color and texture information of the shaft wall surface.
[0016] Secondly, feature calculations are performed on the acquired point cloud images. Specifically, this calculation process mainly extracts two key features: point cloud void ratio and point cloud edge features. The point cloud void ratio is used to quantify the degree of data loss in the 3D point cloud data due to object occlusion or scanning angle limitations; its value directly reflects the completeness of the point cloud data. Point cloud edge features are used to describe abrupt changes in the geometric contour of the well wall surface in 3D space, such as structural joints, component borders, or the boundary lines of surface unevenness.
[0017] Simultaneously, the captured images are processed, specifically, edge features are extracted from images taken from multiple perspectives. Image edge features primarily capture two-dimensional contour lines formed by significant changes in color, brightness, or texture within the captured images. These contour lines may correspond to actual physical edges or originate from surface stains, shadows, or changes in lighting. The edge features from multiple captured images are used to describe the visual contour changes of the well wall surface, characterizing the two-dimensional morphological differences on the well wall surface from different perspectives due to physical structure, attachments, or lighting conditions. These features are then used for cross-modal comparison with point cloud edge features to collaboratively determine the attributes of missing data areas and provide crucial two-dimensional visual evidence for subsequent identification of the spread path and morphology of leaking water.
[0018] Further, registration is performed between the point cloud image and multiple captured images. The registration process establishes an accurate correspondence between the point cloud data in 3D space and the pixels of the captured images. Based on this correspondence, the matching degree between the point cloud edge features and the image edge features is calculated. This matching degree can be specifically calculated by performing spatial consistency analysis on the point cloud edge point set projected onto the image plane and the image edge pixel set extracted by the edge detection operator. For example, it can be based on the overlap of the Hough transform line detection results between the two, or by calculating the proportion of mutually matching edge point pairs within a set distance tolerance. The matching degree directly reflects the consistency between the 3D geometric contour and the 2D visual contour in spatial position. Specifically, in unobstructed visible areas, the point cloud edges highly overlap with the image edges, resulting in a high matching degree; while in areas with occlusion or missing data, the 3D point cloud cannot form complete edges or deviates significantly from the edge cues in the image, leading to a lower matching degree.
[0019] Furthermore, regions with a point cloud void ratio greater than a preset void threshold and a projection matching degree greater than a preset matching threshold are initially marked as potential occlusion areas. A high point cloud void ratio indicates that the 3D point cloud data itself is incomplete, while a high matching degree indicates that the captured image has clear edge features in this region. The contradiction between the missing point cloud data and the complete captured features strongly suggests that there may be foreground occlusions in this region, thus blocking the laser scan but not completely blocking the camera's line of sight, and therefore it is marked as a potential occlusion area.
[0020] Among them, the preset hole threshold is a critical value used to judge the degree of missing point cloud data. It is set according to the statistical analysis of the average data missing ratio caused by typical occlusions in the elevator shaft in the point cloud, for example, in the range of 30% to 50%. The preset matching threshold is a critical value used to measure the consistency between the three-dimensional geometric contour and the two-dimensional visual contour. It is set according to the statistical analysis of the matching degree distribution obtained by conducting a large number of registration experiments on a known unobstructed shaft wall reference area, for example, in the range of 0.7 to 0.9.
[0021] Finally, a more refined joint determination is performed on the initially marked potential occlusion areas. Specifically, this determination process deeply analyzes the sharpness of point cloud edge features and the distribution pattern of image edge features within the potential occlusion area. Specifically, if the sharpness of the point cloud edge corresponding to the potential occlusion area is greater than a second preset threshold, and its corresponding image edge features are discontinuously distributed or have obvious breaks, then the area is determined to have a foreground physical occlusion object and is formally marked as an occlusion area. Conversely, if the sharpness of the point cloud edge corresponding to the potential occlusion area is less than or equal to the second preset threshold, and its corresponding image edge features have a continuous and natural distribution, then the edge is determined to originate from the natural structural boundary of the well wall itself, and the area is excluded from the potential occlusion area. The second preset threshold is a sharpness threshold used to distinguish between physical occlusion edges and natural structural edges. It is set based on statistical learning of a large amount of known point cloud feature data of physical occlusion object edges and natural structural edges of the well wall. For example, it can be set in the range of 0.1 to 0.3 depending on the accuracy and resolution of the point cloud data.
[0022] By introducing a dual discrimination criterion of point cloud edge sharpness and image edge continuity, it is possible to effectively filter out misjudgments caused by noise interference, surface material reflection, or simply poor data quality, thereby accurately identifying the real physical occlusion area and laying the foundation for subsequent inference of the suspected leakage area behind the occlusion.
[0023] S20: Identify leakage in multiple unobstructed areas, obtain multiple leakage areas, and perform feature matching on multiple obstructed areas and multiple leakage areas to obtain multiple suspected leakage areas; Since the occluded areas are the actual shaft wall regions where point cloud data is missing and obscured by foreground objects, while the unoccluded areas are shaft wall surfaces with complete point cloud and image data that can be directly observed, after acquiring multiple occluded and unoccluded areas, it is further necessary to identify leaks in the unoccluded areas. This is to fully utilize the advantage of complete data to accurately detect and locate visible leak traces and their distribution patterns, thereby providing reliable known feature samples and spatial distribution trends for inferring potential hidden leaks in the occluded areas.
[0024] Specifically, leakage is identified in multiple unobstructed areas, resulting in multiple leakage areas, including: Obtain a combined leak detector; The combined leakage detector is used to identify leakage in multiple point cloud images and multiple captured images of multiple unobstructed areas, and to obtain leakage identification results, wherein the leakage identification results include multiple leakage areas and leakage patterns.
[0025] First, a joint leakage identifier is obtained. This joint leakage identifier is a pre-trained analysis model that can fuse point cloud images and captured images. Its internal network structure establishes an end-to-end mapping relationship from multimodal input data to leakage areas and their morphological features, which is used to perform high-confidence leakage identification and accurate segmentation of data-complete, unoccluded areas.
[0026] Specifically, acquiring a combined leak detector includes: Obtain the sample point cloud image set and the corresponding sample image set; The sample point cloud image set and the sample captured image set are labeled to obtain the sample recognition result set; The joint leakage detector is trained using the sample point cloud image set and the sample captured image set as input, and the sample recognition result set as supervision, until convergence. The sample point cloud image set and the sample captured image set are both images of non-occluded areas.
[0027] First, a set of sample point cloud images and corresponding sets of sample captured images are obtained. Both the sample point cloud image set and the sample captured image set are derived from unobstructed areas of different elevator shaft walls to ensure that the training data itself does not suffer from information loss due to occlusion, enabling the joint leakage detector to learn the true representation of leakage water under complete data.
[0028] Secondly, the sample point cloud image set and the sample captured image set are professionally or semi-automatically annotated. It is necessary to accurately identify whether there is a leakage area in each set of data, and further annotate the specific pixel range or three-dimensional spatial location of the leakage area. At the same time, the leakage morphology is classified and delineated to obtain a sample recognition result set containing true value information.
[0029] Finally, the joint leakage identifier is trained using the registered and aligned sample point cloud image set and sample captured image set as input, and the sample recognition result set as the target output, until convergence. For example, the training process terminates when the loss function value of the joint leakage identifier on the validation set no longer decreases significantly for 10 consecutive training epochs, or when its prediction accuracy reaches a preset performance index, such as 95%. At this point, a joint leakage identifier that can be used for practical leakage identification is obtained.
[0030] For example, since the leakage characteristics of the elevator shaft wall are manifested as complex geometric deformation and subtle color and texture changes in point cloud images and captured images, and there is a deep cross-modal correlation between the two, and deep learning models have significant advantages in fusing multi-source heterogeneous data and extracting complex nonlinear features, deep learning models can be selected to construct this joint leakage detector.
[0031] Specifically, this joint leak detector employs a network architecture of dual encoder-decoder-fully connected layer. The dual encoders together constitute the model's feature extraction network layer, processing point cloud images and captured images respectively. The point cloud encoder uses a PointNet++-based point cloud segmentation network structure, whose hierarchical feature learning module effectively captures local geometric anomalies and surface irregularities caused by leaks. The image encoder uses a convolutional neural network structure; its multiple convolutional and pooling layers progressively extract visual features from the bottom edge to the high-level semantics to identify color variations and texture changes caused by water seepage. The multimodal features extracted by the feature extraction network layer are concatenated and fused at the bottleneck layer before being input to the decoder. This decoder consists of a series of transposed convolutional layers, responsible for upsampling the fused high-level features to the original input resolution to generate a spatial feature map. Finally, this spatial feature map is flattened and input to the fully connected layer, which simultaneously outputs the probability that each spatial location belongs to a leaking area and its leak morphology category.
[0032] During training, key hyperparameters included a learning rate of 0.0001, 200 training epochs, and a batch size of 8. The learning rate was set to balance model complexity and training stability, the number of training epochs ensured sufficient convergence of the model on multimodal data, and the batch size was chosen to account for the high memory consumption of point cloud data. Specifically, a supervised learning approach was used, with the previously acquired sample point cloud image set and the corresponding sample image set as input sample pairs, and the sample recognition result set as the supervision signal. All sample data were randomly divided into training, validation, and test sets in an 8:1:1 ratio.
[0033] Furthermore, the sample point cloud images and sample captured images from the training set are used as input, with the corresponding labeled leakage maps and morphology maps as supervision signals. Through backpropagation and the Adam optimizer, all network parameters, including the feature extraction network layer, decoder, and fully connected layers, are iteratively optimized. A combined loss function is employed, consisting of a weighted average of a binary cross-entropy loss for leakage region segmentation and a cross-entropy loss for morphology classification, to jointly optimize both tasks. The training process is continuously monitored using a validation set. When the combined loss function value on the validation set no longer decreases for several consecutive training epochs, and the intersection-over-union (IoU) ratio of the segmentation task reaches a predetermined threshold, such as 0.85, training is terminated, resulting in a converged joint leakage identifyer. This joint leakage identifyer can deeply fuse 3D geometric information and 2D visual information to accurately identify leakage regions in unoccluded areas and determine their morphology.
[0034] Furthermore, in the specific identification process, the point cloud images corresponding to the multiple unobstructed areas and the captured images are jointly input into the joint leakage detector. The joint leakage detector calculates and outputs the leakage identification result. Specifically, the leakage identification result includes not only the spatial location and range of the multiple leakage areas identified within the unobstructed areas, but also a description of the specific leakage morphology exhibited by each leakage area. Specifically, the leakage morphology is a specific depiction of the manifestation of leaked water on the vertical wall surface, which may include various types such as vertical flow marks, localized patchy wetting, and point accumulation.
[0035] The final leakage identification results can provide an accurate data foundation for subsequent feature matching with the occluded area.
[0036] Furthermore, feature matching is performed on the multiple obstructed areas and the multiple leaking areas to obtain multiple suspected leaking areas, including: Obtain the spatial trend matching degree and numerical trend matching degree of multiple said shielding areas and multiple said leakage areas; The spatial trend matching degree and the numerical trend matching degree are weighted and calculated to obtain the comprehensive matching degree, wherein the weights for the weighted calculation are obtained based on the point cloud hole rate; Based on the overall matching degree, areas suspected of leakage are identified.
[0037] Feature matching of multiple obstructed areas and multiple identified leakage areas is a key step in inferring whether there is a hidden leakage behind the obstruction. The multiple suspected leakage areas are obstructed areas that are highly correlated with the identified leakage areas in terms of spatial distribution or feature patterns and are judged to be likely to have hidden leakage. That is, specific obstructed areas that are adjacent to the leakage areas in three-dimensional space, continue in morphological trends, or are similar in physical characteristics.
[0038] Specifically, firstly, it is necessary to obtain the spatial trend matching degree and numerical trend matching degree of multiple shading areas and multiple leakage areas.
[0039] Spatial trend matching is used to assess the consistency between the concealed area and the leakage area in terms of spatial distribution and morphological extension. This spatial trend matching primarily analyzes whether there is a logical continuity in their relative positions, contour directions, and expansion trends within the three-dimensional space of the well wall. For example, if a leakage area extends spatially to the edge of a concealed area, and its water flow trend points inwards from the concealed area, it indicates a higher probability of leakage behind the concealed area, and the spatial trend matching degree between the two increases accordingly.
[0040] Numerical trend matching is used to quantify the similarity of physical features between the occluded and leaking regions. This numerical trend matching is achieved by extracting and comparing the statistical distribution and gradient change patterns of the two regions in terms of quantitative attributes such as point cloud intensity, color saturation, and texture features. If the feature patterns of the edge of the occluded region and the feature patterns of the known leaking region show a high degree of similarity in numerical values, the numerical trend matching is improved accordingly.
[0041] Specifically, obtaining the spatial trend matching degree and numerical trend matching degree of multiple said shading areas and multiple said leakage areas includes: Extract the point cloud connectivity index and point cloud intensity gradient of multiple point cloud images of the occluded regions, and extract the gray-level mean gradient and saturation abrupt change coefficient of multiple images of the occluded regions. Based on the point cloud intensity gradient, the point cloud intensity anomaly trends of multiple occluded regions are obtained; Based on the gray-level mean gradient, obtain the gray-level gradient anomaly trends of multiple occluded regions; Determine the sign consistency between the abnormal trend of point cloud intensity and the abnormal trend of gray-level gradient, assign a consistency coefficient, and obtain the spatial trend matching degree by combining the matching degree of point cloud edge features and image edge features. The numerical trend matching degree is obtained by combining the point cloud connectivity index and the saturation mutation coefficient.
[0042] First, for the point cloud image of the occluded area, the point cloud connectivity index and point cloud intensity gradient are calculated. The point cloud connectivity index characterizes the morphological orientation of continuous point clouds within the target area, specifically defined as the ratio of the vertical extension length to the horizontal diffusion width of the point cloud cluster in that area. This ratio is calculated using a point cloud clustering algorithm and can reflect the vertical water flow traces that may be caused by seepage. The point cloud intensity gradient describes the rate of change of the reflection intensity of the point cloud around the occluded area along the detection direction. It is calculated by differentiating the intensity value and can be used to indicate changes in surface humidity.
[0043] Simultaneously, for images captured in the same occluded area, the gray-level mean gradient and saturation abrupt change coefficient are extracted. The gray-level mean gradient reflects the radial rate of change of the gray-level mean of the surrounding image in the occluded area, and is obtained through pixel statistics and differential calculation. The saturation abrupt change coefficient is used to capture abnormal changes in surface color saturation caused by leakage, and is specifically defined as the absolute value of the saturation difference between the occluded area and the adjacent normal material area identified by semantic segmentation technology.
[0044] Furthermore, based on the direction of change in the point cloud intensity gradient, abnormal trends in point cloud intensity are determined. When the direction of the point cloud intensity gradient points towards the interior of the occluded area, it is defined as a positive trend, indicating a possible increase in humidity; conversely, it is a negative trend. Similarly, based on the direction of change in the grayscale mean gradient, abnormal trends in grayscale gradient are determined. When the direction of the grayscale mean gradient points towards the interior of the occluded area, it is defined as a positive trend, indicating a possible darkening of color; conversely, it is a negative trend.
[0045] Furthermore, the sign consistency between the abnormal trend of point cloud intensity and the abnormal trend of grayscale gradient is assessed. If their directions of change are consistent, a higher first consistency coefficient is assigned, ranging from 0.7 to 1.0; if their directions of change are inconsistent, a lower second consistency coefficient is assigned, ranging from 0 to 0.3. The consistency coefficient is a weighted factor used to quantify the degree of directional consistency between the trend of 3D point cloud intensity change and the trend of 2D image grayscale change. It characterizes the strength of synergistic evidence provided by multimodal sensing data in indicating potential leakage anomalies, and is used to enhance or weaken the confidence of conclusions based on a single data source in spatial trend analysis. The specific values of the first and second consistency coefficients are set based on the statistical distribution of the probability of occurrence of verified leakage cases in historical data under conditions of consistent and inconsistent trends in different modal data.
[0046] Finally, the consistency coefficient is combined with the matching degree of point cloud edge features and image edge features obtained from the previous joint occlusion analysis to form the spatial trend matching degree. The spatial trend matching degree is equal to the product of the consistency coefficient and the edge feature matching degree. This calculation method characterizes the completeness and reliability of the cross-modal evidence chain from geometric contour alignment to the trend of physical attribute changes. It is used to ultimately comprehensively evaluate the strength of the spatial development correlation between the occluded area and the known leakage area, providing key spatial correlation basis for screening suspected leakage areas.
[0047] Furthermore, the calculation of numerical trend matching focuses on the direct correlation between physical and optical features, obtained by combining the point cloud connectivity index and the saturation abrupt change coefficient. The numerical trend matching degree is equal to the product of the point cloud connectivity index and the saturation abrupt change coefficient. A higher point cloud connectivity index indicates a more pronounced vertical extension feature, which better matches the leakage feature; a larger saturation abrupt change coefficient indicates a more significant color anomaly. Therefore, the numerical trend matching degree can effectively amplify the matching signal corresponding to regions exhibiting significant anomalies in both point cloud morphology and image color, characterizing the joint probability strength of occluded regions simultaneously exhibiting leakage-related anomalies in both physical structural deformation and surface optical properties.
[0048] Furthermore, after obtaining the spatial trend matching degree and the numerical trend matching degree respectively, they are weighted and calculated to obtain the comprehensive matching degree. Specifically, the weight allocation of the weighted calculation dynamically depends on the point cloud hole rate of the corresponding occluded region. The higher the point cloud hole rate, the more thoroughly the region is occluded, and the less reliable the three-dimensional geometric information. In this case, the numerical trend matching degree should be given higher weight in the calculation, because the two-dimensional image features may be more critical; conversely, the calculation may rely more on the spatial trend matching degree.
[0049] For example, when the point cloud void rate in a certain occluded area is higher than 70%, a weight of 0.7 can be assigned to the numerical trend matching degree and a weight of 0.3 to the spatial trend matching degree; when the point cloud void rate is between 30% and 70%, equal weights can be assigned to both, i.e., 0.5 each; when the point cloud void rate is lower than 30%, a weight of 0.7 can be assigned to the spatial trend matching degree and a weight of 0.3 to the numerical trend matching degree. This dynamic weight allocation mechanism can adaptively balance the contributions of different reliable data sources, ensuring the robustness of the comprehensive matching degree evaluation results.
[0050] The calculated comprehensive matching degree characterizes the strength of the overall correlation between a specific obstructed area and a known leaking area in terms of spatial distribution, morphological trends, and physical-optical characteristics. This serves as a unified quantitative basis for subsequent screening of suspected leaking areas. Finally, screening is performed based on the calculated comprehensive matching degree. Specifically, a comprehensive matching degree threshold is set. This threshold is a critical value used to determine whether the correlation is significant. It is set according to the statistical characteristics of the matching degree distribution between actual leaking areas and adjacent obstructed areas in historical verification data, for example, 0.6. When the comprehensive matching degree of an obstructed area is greater than or equal to this threshold, the area is determined to be a suspected leaking area; when the comprehensive matching degree is less than the threshold, the area is excluded from the suspected area range. Finally, multiple suspected leaking areas with high leakage risk are accurately screened from all obstructed areas.
[0051] S30: Based on the well wall material, perform leakage verification on multiple suspected leakage areas and obtain multiple leakage verification results, wherein the leakage verification results include leakage probability and leakage pattern; Specifically, based on the well wall material, multiple suspected leakage areas are verified for leakage, and multiple leakage verification results are obtained. These results include leakage probability and leakage pattern, including: Obtain the well wall material of multiple suspected leakage areas; Based on the shaft wall material, obtain the material influence factor; Transfer learning is performed on the joint leak detector to obtain the leak verifier; The leakage verification device is used to verify the leakage in multiple suspected leakage areas and obtain multiple leakage verification results, wherein the leakage verification results include leakage probability and leakage morphology.
[0052] First, it is necessary to determine the shaft wall material corresponding to each suspected leakage area, such as concrete, sprayed steel plate, or stainless steel. After obtaining the material information, the corresponding material influence factor is obtained based on the material. The material influence factor is used to characterize the degree of water adsorption, diffusion, and the significance of features presented in images and point clouds when different material surfaces undergo leakage. This material influence factor is obtained by querying a mapping table, which maps the quantitative or qualitative correspondence between common shaft wall material types and their typical physical characteristics and optical response features exhibited under leakage conditions. For example, concrete material, due to its porous nature, results in a large water stain diffusion range but low color contrast, and its material influence factor may focus on morphological analysis; while stainless steel material may exhibit obvious water droplet accumulation and high light reflection, and its material influence factor focuses more on reflection feature analysis.
[0053] Furthermore, transfer learning is applied to the trained joint leakage detector to construct a dedicated leakage validator. The transfer learning process retains the general feature extraction capabilities learned by the joint leakage detector in unoccluded areas, while fine-tuning its network structure to incorporate material influence factors and adapt to the incomplete data characteristics of occluded areas.
[0054] Specifically, transfer learning is performed on the joint leakage identifier to obtain a leakage verifier, including: Freeze the feature extraction network layer of the combined leakage detector; Add a material influence factor adaptation layer before the fully connected layer of the joint leakage detector; Obtain the sample occlusion point cloud image set, sample occlusion image set, and corresponding occlusion material influence factor set with occlusion areas, and annotate them. The annotation content is the leakage verification result, and obtain the leakage verification result set. Using the sample occlusion point cloud image set, the sample occlusion captured image set, and the occlusion material influence factor set as the fine-tuning input dataset, and the leakage verification result set as the fine-tuning verification result set, the parameters of the adaptation layer and subsequent network layers are trained and fine-tuned until the loss function converges, thereby obtaining the leakage verifier.
[0055] First, the feature extraction network layer of the joint leak detector is frozen. Freezing means keeping the weight parameters of the feature extraction network layer unchanged in subsequent training. The purpose is to preserve the general recognition ability of the joint leak detector on the basic characteristics of leaks learned from the data of unobstructed areas, and to avoid catastrophic forgetting of the joint leak detector due to training with limited data samples from obstructed areas.
[0056] Secondly, a novel material influence factor adaptation layer is added before the fully connected layer of the joint leakage detector. This material influence factor adaptation layer is a small neural network structure whose core function is to deeply fuse and interact with the externally input material influence factors and the multimodal features output by the feature extraction network, enabling the joint leakage detector to dynamically adjust its interpretation and weight allocation of input features according to the material characteristics.
[0057] Next, a dedicated dataset for fine-tuning training is prepared. Specifically, a set of sample occluded point cloud images, a set of sample occluded images, and their corresponding set of occlusion material influence factors are obtained. This set of sample occluded point cloud images and sample occluded images needs to be professionally annotated with actual leakage verification results, such as whether leakage actually exists in the occluded area and the specific leakage pattern, thus forming a leakage verification result set.
[0058] Finally, fine-tuning training was performed using a set of sample occlusion point cloud images, a set of sample occlusion captured images, and a set of occlusion material influence factors as the fine-tuning input dataset, with the leakage verification result set as the supervision target. During training, only the parameters of the newly added material influence factor adaptation layer and the fully connected layers and subsequent network layers in the original joint leakage detector were updated. Through iterative optimization, the joint leakage detector learned to make accurate leakage judgments while considering the influence of materials and the incompleteness of occlusion data. Training stopped when the loss function converged, and the optimized model obtained at this point became the leakage verifier specifically for verifying suspected leakage areas.
[0059] Finally, the leakage verifier was used to verify multiple suspected leakage areas. The leakage verifier takes point cloud images, captured images, and material influence factors of the suspected leakage areas as comprehensive inputs, and outputs the leakage verification results for each suspected leakage area after internal calculations. These results include leakage probability and leakage morphology. Leakage probability indicates the likelihood of actual leakage in the suspected area, while leakage morphology describes the specific distribution and manifestation pattern of water infiltration and flow under specific conditions on a vertical wall, such as vertical strip-like flow, localized patchy dampness, point accumulation, or network diffusion. The obtained multiple leakage verification results can provide comprehensive information, including location, probability, and manifestation, for subsequent maintenance decisions.
[0060] S40: Integrate multiple leakage verification results and multiple leakage areas to obtain an elevator shaft wall leakage identification report and output it.
[0061] Specifically, by integrating multiple leakage verification results and multiple leakage areas, an elevator shaft wall leakage identification report is obtained and output, including: The multiple leakage verification results and leakage areas are structured and integrated to obtain an elevator shaft wall leakage identification report, wherein the elevator shaft wall leakage identification report includes at least the leakage area, leakage pattern and leakage probability. Output the water leakage identification report of the elevator shaft wall.
[0062] First, the leakage areas identified by the joint leakage detector in unobstructed areas are summarized with the leakage verification results obtained by the leakage verifier after verifying the covered areas, and stored in a unified database or structured data model. Based on this integrated data, a complete elevator shaft wall leakage identification report is generated. Specifically, the elevator shaft wall leakage identification report covers at least three core components: first, the spatial information of all identified leakage areas, clarifying their specific location and distribution range in the three-dimensional space of the shaft; second, the morphological characteristics corresponding to each leakage area, specifically described as typical patterns such as vertical strip-like flow and localized patchy dampness; and third, the leakage probability of each leakage area, which is used to quantitatively assess the confidence level of leakage and provide a basis for risk level classification.
[0063] Finally, the generated elevator shaft wall leakage identification report is output. Output formats include, but are not limited to, visual electronic documents, annotation files integrated into a 3D model, or structured data tables that can be directly accessed by the maintenance management system. This elevator shaft wall leakage identification report provides comprehensive, accurate, and actionable decision support for targeted maintenance and leakage control of elevator shafts.
[0064] In summary, the embodiments of this application have at least the following technical effects: Compared to existing technologies, this application firstly uses joint occlusion analysis of 3D point clouds and 2D images to accurately distinguish between occluded and unoccluded areas of the elevator shaft wall, effectively overcoming detection blind spots caused by the complex structure within the shaft and improving the comprehensiveness of detection coverage. Secondly, based on accurate leakage identification in unoccluded areas, it innovatively performs multi-feature matching between the identification results and occluded areas to infer potential leakage suspicion areas, achieving collaborative detection of visible and invisible areas and significantly reducing the risk of missed detections. Thirdly, addressing the identification challenges brought about by the diversity of shaft wall materials, it introduces material influence factors and employs transfer learning technology to construct a dedicated validator, effectively improving the accuracy and generalization ability of leakage identification on different material surfaces and reducing false positives. Finally, by structurally integrating all identification and verification results, a comprehensive detection report including location, morphology, and probability is generated, providing intuitive, reliable, and quantifiable data support for subsequent maintenance decisions, thus achieving a unity of comprehensiveness, accuracy, and practicality in elevator shaft wall leakage detection.
[0065] Example 2, as Figure 2As shown, based on the same inventive concept as the elevator shaft wall leakage identification method based on 3D point cloud and image provided in Embodiment 1, this embodiment of the invention also provides an elevator shaft wall leakage identification system based on 3D point cloud and image, including: The data acquisition and joint analysis module 11 is used to acquire point cloud images and captured images of the elevator shaft wall area to be identified, and to perform joint occlusion analysis to acquire multiple occluded areas and multiple non-occluded areas. The leakage identification and feature matching module 12 is used to identify leakage in multiple unobstructed areas, obtain multiple leakage areas, and perform feature matching on multiple obstructed areas and multiple leakage areas to obtain multiple suspected leakage areas. Leakage verification module 13 is used to verify the leakage of multiple suspected leakage areas in combination with the well wall material and obtain multiple leakage verification results, wherein the leakage verification results include leakage probability and leakage mode. The report generation and output module 14 is used to integrate multiple leakage verification results and multiple leakage areas to obtain an elevator shaft wall leakage identification report and output it.
[0066] Specifically, the data acquisition and joint analysis module 11 is used for: Obtain point cloud images and captured images of the elevator shaft wall region to be identified, and perform joint occlusion analysis to obtain multiple occlusion regions, including: Acquire point cloud images and captured images of the elevator shaft wall area to be identified, and perform point cloud image feature calculation to obtain point cloud void ratio and point cloud edge features; Obtain the image edge features of multiple captured images; The point cloud image and multiple captured images are registered to obtain the matching degree between the edge features of the point cloud and the edge features of the image. Regions with a point cloud hole rate greater than a preset hole threshold and a projection matching degree greater than a preset matching threshold are marked as potential occlusion areas. The point cloud edge features and image edge features of the potential occlusion area are jointly determined to filter and obtain the occlusion area.
[0067] The leakage identification and feature matching module 12 is specifically used for: Leakage identification was performed in multiple unobstructed areas, resulting in multiple leakage areas, including: Obtain a combined leak detector; The combined leakage detector is used to identify leakage in multiple point cloud images and multiple captured images of multiple unobstructed areas, and to obtain leakage identification results, wherein the leakage identification results include multiple leakage areas and leakage patterns.
[0068] First, obtain the combined leak detector, including: Obtain the sample point cloud image set and the corresponding sample image set; The sample point cloud image set and the sample captured image set are labeled to obtain the sample recognition result set; The joint leakage detector is trained using the sample point cloud image set and the sample captured image set as input, and the sample recognition result set as supervision, until convergence. The sample point cloud image set and the sample captured image set are both images of non-occluded areas.
[0069] Furthermore, feature matching is performed on the multiple obstructed areas and the multiple leaking areas to obtain multiple suspected leaking areas, including: Obtain the spatial trend matching degree and numerical trend matching degree of multiple said shielding areas and multiple said leakage areas; The spatial trend matching degree and the numerical trend matching degree are weighted and calculated to obtain the comprehensive matching degree, wherein the weights for the weighted calculation are obtained based on the point cloud hole rate; Based on the overall matching degree, areas suspected of leakage are identified.
[0070] Specifically, obtaining the spatial trend matching degree and numerical trend matching degree of multiple said shading areas and multiple said leakage areas includes: Extract the point cloud connectivity index and point cloud intensity gradient of multiple point cloud images of the occluded regions, and extract the gray-level mean gradient and saturation abrupt change coefficient of multiple images of the occluded regions. Based on the point cloud intensity gradient, the point cloud intensity anomaly trends of multiple occluded regions are obtained; Based on the gray-level mean gradient, obtain the gray-level gradient anomaly trends of multiple occluded regions; Determine the sign consistency between the abnormal trend of point cloud intensity and the abnormal trend of gray-level gradient, assign a consistency coefficient, and obtain the spatial trend matching degree by combining the matching degree of point cloud edge features and image edge features. The numerical trend matching degree is obtained by combining the point cloud connectivity index and the saturation mutation coefficient.
[0071] The leakage verification module 13 is specifically used for: Based on the wellbore wall material, multiple suspected leakage areas were verified for leakage, and multiple leakage verification results were obtained. These results include leakage probability and leakage morphology, including: Obtain the well wall material of multiple suspected leakage areas; Based on the shaft wall material, obtain the material influence factor; Transfer learning is performed on the joint leak detector to obtain the leak verifier; The leakage verification device is used to verify the leakage in multiple suspected leakage areas and obtain multiple leakage verification results, wherein the leakage verification results include leakage probability and leakage morphology.
[0072] Specifically, transfer learning is performed on the joint leakage identifier to obtain a leakage verifier, including: Freeze the feature extraction network layer of the combined leakage detector; Add a material influence factor adaptation layer before the fully connected layer of the joint leakage detector; Obtain the sample occlusion point cloud image set, sample occlusion image set, and corresponding occlusion material influence factor set with occlusion areas, and annotate them. The annotation content is the leakage verification result, and obtain the leakage verification result set. Using the sample occlusion point cloud image set, the sample occlusion captured image set, and the occlusion material influence factor set as the fine-tuning input dataset, and the leakage verification result set as the fine-tuning verification result set, the parameters of the adaptation layer and subsequent network layers are trained and fine-tuned until the loss function converges, thereby obtaining the leakage verifier.
[0073] The report generation and output module 14 is specifically used for: By integrating multiple leakage verification results and multiple leakage areas, an elevator shaft wall leakage identification report is obtained and output, including: The multiple leakage verification results and leakage areas are structured and integrated to obtain an elevator shaft wall leakage identification report, wherein the elevator shaft wall leakage identification report includes at least the leakage area, leakage pattern and leakage probability. Output the water leakage identification report of the elevator shaft wall.
[0074] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.
[0075] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.
[0076] This specification and accompanying drawings are merely illustrative examples of this application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Therefore, if such modifications and modifications fall within the scope of this application and its equivalents, this application intends to include such modifications and modifications.
Claims
1. A method for identifying water leakage in elevator shaft walls based on 3D point clouds and images, characterized in that, include: The point cloud image and captured image of the elevator shaft wall area to be identified are obtained, and joint occlusion analysis is performed to obtain multiple occluded areas and multiple non-occluded areas. Leakage is identified in multiple unobstructed areas to obtain multiple leakage areas, and feature matching is performed on multiple obstructed areas and multiple leakage areas to obtain multiple suspected leakage areas; Based on the well wall material, multiple suspected leakage areas are verified for leakage, and multiple leakage verification results are obtained. The leakage verification results include leakage probability and leakage pattern. By integrating multiple leakage verification results and multiple leakage areas, an elevator shaft wall leakage identification report is obtained and output.
2. The method for identifying water leakage in elevator shaft walls based on three-dimensional point clouds and images according to claim 1, characterized in that, Obtain point cloud images and captured images of the elevator shaft wall region to be identified, and perform joint occlusion analysis to obtain multiple occlusion regions, including: Acquire point cloud images and captured images of the elevator shaft wall area to be identified, and perform point cloud image feature calculation to obtain point cloud void ratio and point cloud edge features; Obtain the image edge features of multiple captured images; The point cloud image and multiple captured images are registered to obtain the matching degree between the edge features of the point cloud and the edge features of the image. Regions with a point cloud hole rate greater than a preset hole threshold and a projection matching degree greater than a preset matching threshold are marked as potential occlusion areas. The point cloud edge features and image edge features of the potential occlusion area are jointly determined to filter and obtain the occlusion area.
3. The method for identifying water leakage in elevator shaft walls based on three-dimensional point clouds and images according to claim 1, characterized in that, Leakage identification was performed in multiple unobstructed areas, resulting in multiple leakage areas, including: Obtain a combined leak detector; The combined leakage detector is used to identify leakage in multiple point cloud images and multiple captured images of multiple unobstructed areas, and to obtain leakage identification results, wherein the leakage identification results include multiple leakage areas and leakage patterns.
4. The method for identifying water leakage in elevator shaft walls based on three-dimensional point clouds and images according to claim 1, characterized in that, Acquiring a combined leak detector includes: Obtain the sample point cloud image set and the corresponding sample image set; The sample point cloud image set and the sample captured image set are labeled to obtain the sample recognition result set; The joint leakage detector is trained using the sample point cloud image set and the sample captured image set as input, and the sample recognition result set as supervision, until convergence. The sample point cloud image set and the sample captured image set are both images of non-occluded areas.
5. The method for identifying water leakage in elevator shaft walls based on three-dimensional point clouds and images according to claim 1, characterized in that, Feature matching is performed on multiple obstructed areas and multiple leaking areas to obtain multiple suspected leaking areas, including: Obtain the spatial trend matching degree and numerical trend matching degree of multiple said shielding areas and multiple said leakage areas; The spatial trend matching degree and the numerical trend matching degree are weighted and calculated to obtain the comprehensive matching degree, wherein the weights for the weighted calculation are obtained based on the point cloud hole rate; Based on the overall matching degree, areas suspected of leakage are identified.
6. The method for identifying water leakage in elevator shaft walls based on three-dimensional point clouds and images according to claim 5, characterized in that, Obtaining the spatial trend matching degree and numerical trend matching degree of multiple said shading areas and multiple said leakage areas, including: Extract the point cloud connectivity index and point cloud intensity gradient of multiple point cloud images of the occluded regions, and extract the gray-level mean gradient and saturation abrupt change coefficient of multiple images of the occluded regions. Based on the point cloud intensity gradient, the point cloud intensity anomaly trends of multiple occluded regions are obtained; Based on the gray-level mean gradient, obtain the gray-level gradient anomaly trends of multiple occluded regions; Determine the sign consistency between the abnormal trend of point cloud intensity and the abnormal trend of gray-level gradient, assign a consistency coefficient, and obtain the spatial trend matching degree by combining the matching degree of point cloud edge features and image edge features. The numerical trend matching degree is obtained by combining the point cloud connectivity index and the saturation mutation coefficient.
7. The method for identifying water leakage in elevator shaft walls based on three-dimensional point clouds and images according to claim 1, characterized in that, Based on the wellbore wall material, multiple suspected leakage areas were verified for leakage, and multiple leakage verification results were obtained. These results include leakage probability and leakage morphology, including: Obtain the well wall material of multiple suspected leakage areas; Based on the shaft wall material, obtain the material influence factor; Transfer learning is performed on the joint leak detector to obtain the leak verifier; The leakage verification device is used to verify the leakage in multiple suspected leakage areas and obtain multiple leakage verification results, wherein the leakage verification results include leakage probability and leakage morphology.
8. The method for identifying water leakage in elevator shaft walls based on three-dimensional point clouds and images according to claim 1, characterized in that, Transfer learning is performed on the joint leak detector to obtain a leak verifier, including: Freeze the feature extraction network layer of the combined leakage detector; Add a material influence factor adaptation layer before the fully connected layer of the joint leakage detector; Obtain the sample occlusion point cloud image set, sample occlusion image set, and corresponding occlusion material influence factor set with occlusion areas, and annotate them. The annotation content is the leakage verification result, and obtain the leakage verification result set. Using the sample occlusion point cloud image set, the sample occlusion captured image set, and the occlusion material influence factor set as the fine-tuning input dataset, and the leakage verification result set as the fine-tuning verification result set, the parameters of the adaptation layer and subsequent network layers are trained and fine-tuned until the loss function converges, thereby obtaining the leakage verifier.
9. The method for identifying water leakage in elevator shaft walls based on three-dimensional point clouds and images according to claim 1, characterized in that, By integrating multiple leakage verification results and multiple leakage areas, an elevator shaft wall leakage identification report is obtained and output, including: The multiple leakage verification results and leakage areas are structured and integrated to obtain an elevator shaft wall leakage identification report, wherein the elevator shaft wall leakage identification report includes at least the leakage area, leakage pattern and leakage probability. Output the water leakage identification report of the elevator shaft wall.
10. A water leakage identification system for elevator shaft walls based on 3D point cloud and image, characterized in that, The method for identifying water leakage in elevator shaft walls based on three-dimensional point clouds and images, as described in any one of claims 1-9, includes: The data acquisition and joint analysis module is used to acquire point cloud images and captured images of the elevator shaft wall area to be identified, and to perform joint occlusion analysis to acquire multiple occluded areas and multiple unoccluded areas. The leakage identification and feature matching module is used to identify leakage in multiple unobstructed areas, obtain multiple leakage areas, and perform feature matching on multiple obstructed areas and multiple leakage areas to obtain multiple suspected leakage areas. The leakage verification module is used to verify the leakage of multiple suspected leakage areas in combination with the well wall material, and obtain multiple leakage verification results, wherein the leakage verification results include leakage probability and leakage mode. The report generation and output module is used to integrate multiple leakage verification results and multiple leakage areas to obtain an elevator shaft wall leakage identification report and output it.