Method, system and equipment for automatically detecting damage of whole wall surface based on video stream

Abstract

The invention discloses a method, a system and equipment for automatically detecting integral wall surface damage based on a video stream. The method comprises the steps of model training, video dynamic acquisition, regional wall surface image sequence extraction, wall surface panoramic image fusion and automatic wall surface damage detection. Adaptively segmenting the wall panoramic image into a wall block image sequence through the trained instance segmentation neural network model, and automatically detecting and marking the wall damage condition; and splicing all the wall surface block images after automatic detection and labeling to form a wall surface damage panoramic image, and performing overall overview display and detail drawing damage information labeling display on the wall surface damage position distribution condition of the whole wall surface through the wall surface damage panoramic image. According to the invention, automatic detection and intelligent detection of the damage condition of the whole wall surface can be realized, the method has the advantages of high detection efficiency and high intelligent level, and has double display effects of global display and display of the detail drawing damage information of the damage position node.

Description

Technical Field

[0001] The present invention relates to the field of existing building maintenance, and in particular to a method, system and equipment for automatic detection of integral wall damage based on video stream. Background Art

[0002] Existing buildings, especially historical buildings, will suffer varying degrees of damage as they age, among which wall pollution, cracking, defects, weathering and other damage are the most common. Due to the high value of historical buildings, the identification and diagnosis of wall damage is an important part of the preventive protection of historical buildings and the basis for establishing scientific and effective maintenance methods. Traditional superficial damage identification and diagnosis relies on irregular inspections and annotations by professional technicians. The work is delayed, mainly based on qualitative descriptions, prone to missed inspections, and low efficiency. For wide walls, vertical walls and other integral walls that are continuous in a certain direction, it is impossible to manually detect and mark damage to the integral wall due to the upper limit of height, length or width. Therefore, digital technology is used to identify and diagnose the damage to the integral wall. The disadvantage is that the integral wall needs to be divided into areas and photographed to form several wall area pictures, and there needs to be a certain overlap rate between adjacent wall area pictures to fully cover the entire wall. The damage in the wall area pictures is manually detected, identified and marked, resulting in low efficiency in detecting the damage to the integral wall. Due to the overlap rate problem between adjacent wall area pictures, when the wall area pictures are manually spliced, the splicing workload is large, the splicing efficiency is low, and visual fatigue easily leads to splicing errors, resulting in the damage of each wall area picture cannot be accurately located and reflected in the corresponding position of the integral wall. Summary of the invention

[0003] The purpose of the present invention is to provide a method, system and equipment for automatic detection of overall wall damage based on video stream, so as to solve the problems in the prior art of low efficiency in detecting and identifying damage to an overall wall that is continuously distributed in at least one direction and difficulty in matching and locating the damage position on the overall wall.

[0004] In order to solve the above technical problems, the present invention provides a method for automatic detection of overall wall damage based on video stream, comprising: Model training: Use the wall images that have been manually inspected for damage as sample data to train the automatic detection and automatic labeling of wall image damage through the instance segmentation neural network model; Video dynamic acquisition: parallel to the existing building wall, collect the entire wall video image of a certain wall of the existing building; Regional wall image sequence extraction: extract the whole wall video image into a continuous regional wall image sequence; Wall panoramic image fusion: Extract feature points of wall images in each area, match feature points of wall images in adjacent areas, and fuse all area wall images into a wall panoramic image covering the entire wall; Automatic detection of wall damage: The trained instance segmentation neural network model is used to adaptively segment the wall panoramic image into a continuous wall block image sequence according to the pixel size compressed by the neural network, and the wall damage of each wall block image is automatically detected and labeled; all the wall block images after automatic detection and automatic labeling are spliced ​​to form a wall damage panoramic image covering the entire wall surface. The wall damage panoramic image can be used to provide an overall overview of the distribution of wall damage locations on the entire wall of the existing building, and the damage information of each wall damage location is annotated and displayed in detail.

[0005] Furthermore, in the automatic detection method of overall wall damage based on video stream provided by the present invention, the instance segmentation neural network model is Mask R-CNN, YOLACT or an improved version of YOLO.

[0006] Furthermore, the present invention provides a method for automatically detecting overall wall damage based on video streams, wherein the method for extracting regional wall image sequences comprises: Using video coding and decoding technology, through video decompression, color space conversion, image encoding and timestamp management, the key frame extraction interval of the overall wall video image is calculated by an adaptive key frame selection algorithm, and the regional wall image is extracted from the overall wall video image by rounding down the frame number interval, so as to decompose the overall wall video image into a continuous regional wall image sequence.

[0007] Furthermore, the video stream-based overall wall damage automatic detection method provided by the present invention further includes, after the regional wall image sequence is extracted and before the wall panoramic image is fused: Wall image preprocessing: including: Image enhancement: For wall area images with uneven lighting and low contrast, histogram equalization and adaptive histogram equalization techniques are used to locally and adaptively enhance the image contrast and highlight the wall texture and damage features of the wall area images. Noise reduction processing: Guided filtering and / or median filtering methods are used to perform graded noise reduction on the wall area image after image enhancement, so as to retain its damage characteristics while removing the noise of the wall area image; Image correction: For distorted wall area images, feature points are identified in the wall area images and the wall area images are corrected using perspective transformation and affine transformation algorithms so that the wall appears as a regular rectangle in the wall area images.

[0008] Furthermore, the present invention provides a method for automatically detecting overall wall damage based on video streams, wherein the method for fusing the wall panoramic images comprises: Feature extraction: Scale-invariant feature transformation and accelerated robust feature algorithm are used to extract feature points and feature descriptions in the wall area image; Feature matching: A matching algorithm based on distance measurement is used to match the feature points of adjacent wall area images; Registration and fusion: Based on the matched feature points, the wall area images are registered using affine transformation and homography transformation methods to align the same areas of adjacent wall area images in space. By calculating the transformation matrix, the wall area images are transformed into a unified coordinate system, and a higher fusion weight is given to the overlapping central part of the wall area images, and a lower fusion weight is given to the edge part, so that the transition of the fused wall area images is natural, forming a complete panoramic image of the wall.

[0009] Furthermore, in the automatic detection method for overall wall damage based on video stream provided by the present invention, in the step of feature matching: for the matching results, the model parameters are calculated and verified through a random sampling consistency algorithm, and the mismatching points are removed to improve the matching accuracy.

[0010] In order to solve the above technical problems, the present invention further provides a system for automatically detecting overall wall damage based on video stream, which adopts the above-mentioned method for automatically detecting overall wall damage based on video stream, and comprises: A model training module is used to use the wall images that have been manually inspected for damage as sample data to train the automatic detection and automatic labeling of the wall image damage through the instance segmentation neural network model; The video dynamic acquisition module is used to acquire the whole wall video image of a certain wall of an existing building in parallel with the wall of the existing building; A regional wall image sequence extraction module is used to extract the entire wall video image into a continuous regional wall image sequence; The wall panoramic image fusion module is used to extract the feature points of the wall images of each area, match the feature points of the wall images of adjacent areas, and fuse the wall images of all areas into a wall panoramic image covering the entire wall; The automatic wall damage detection module is used to adaptively segment the wall panoramic image into a continuous wall block image sequence according to the pixel size compressed by the neural network through the trained instance segmentation neural network model, and automatically detect and automatically mark the wall damage of each wall block image; all wall block images after automatic detection and automatic marking are spliced ​​to form a wall damage panoramic image covering the entire wall surface, and the wall damage panoramic image is used to provide an overall overview of the distribution of wall damage positions on the entire wall of the existing building and to provide detailed annotations of the damage information of each wall damage position.

[0011] In order to solve the above technical problems, the present invention also provides an automatic detection device for integral wall damage based on video stream, which adopts the above automatic detection system for integral wall damage based on video stream.

[0012] Furthermore, the video stream-based automatic detection system for overall wall damage provided by the present invention further includes: The wall image preprocessing module is used to perform image enhancement, noise reduction and image correction on the wall area image.

[0013] Furthermore, the present invention provides an automatic detection device for overall wall damage based on video stream, and the device is a drone, a ground robot or a wall-climbing robot equipped with a high-definition camera.

[0014] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a method, system and device for automatically detecting overall wall damage based on video stream, which collects video images of the overall wall surface of a certain wall of an existing building wall, extracts regional wall surface image sequences from the collected video images, fuses the regional wall surface image sequences into a wall surface panoramic image, automatically segments the wall surface panoramic image into a wall surface block image sequence through a trained instance segmentation neural network model, and automatically detects and automatically labels the wall surface damage conditions, and splices all the wall surface block images after automatic detection and automatic labeling to form a wall surface damage image covering the overall wall surface. The panoramic image of the wall damage can provide an overall overview of the distribution of the wall damage positions of the entire wall of the existing building and provide detailed annotation of the damage information of each wall damage position, thereby realizing the automatic detection and intelligent detection of the overall wall damage of the existing building's wide wall, vertical wall and other walls that have continuity in a certain direction. It can achieve the dual display effects of global display of the distribution of the damage positions of the entire wall of the existing building and detailed display of the damage position nodes, so that the damage position and its damage information can be matched to avoid confusion caused by mismatch between the two. Compared with manual detection of damage conditions on wall entities and manual detection and annotation of damage conditions on a large number of wall area pictures, it has the advantages of high detection efficiency and high intelligence level, and can provide guidance for the preventive protection of historical buildings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a flow chart of a method for automatic detection of overall wall damage based on video streams according to an embodiment; Figure 2 is a flow chart of a method for automatic detection of overall wall damage based on video streams according to another embodiment; Figure 3 It is a block diagram of the overall wall damage automatic detection system based on video stream; Figure 4 It is a schematic diagram of the structure of the automatic detection device for overall wall damage based on video stream; As shown in the figure: 10. Automatic detection equipment for overall wall damage based on video stream, 11. Drone, 12. Ground robot, 13. Wall climbing robot; 100. Automatic detection system for overall wall damage based on video stream, 110. Model training module, 120. Video dynamic acquisition module, 130. Regional wall image sequence extraction module, 140. Wall image preprocessing module, 150. Wall panoramic image fusion module, 160. Automatic wall damage detection module. DETAILED DESCRIPTION

[0016] The present invention is described in detail below in conjunction with the accompanying drawings: The advantages and features of the present invention will become more apparent from the following description. It should be noted that the accompanying drawings are in very simplified form and in non-precise proportions, and are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.

[0017] Please refer to Figure 1 to Figure 2 The embodiment of the present invention provides a method for automatically detecting overall wall damage based on video stream, comprising: Step S1, model training. The wall images that have been manually inspected for damage are used as sample data to perform automatic detection and automatic labeling of wall image damage through instance segmentation neural network models such as MaskR-CNN, YOLACT, or ‌YOLO improved version. The sample data includes wall images with manual labeling of damage and unlabeled wall images without damage. The sample data needs to collect a large number of wall images of existing buildings such as historical buildings with different types of damage and degrees of damage (such as cracks, weathering), manually detect and label the damage of the wall images, and mark the damaged areas and damage information.

[0018] Step S2, video dynamic acquisition. Parallel to the wall of the existing building, the whole wall video image of a certain wall of the existing building is acquired. The existing building can be a historical building or a modern building in use. During the acquisition of the whole wall video image of the existing building, sufficient and uniform light should be ensured to ensure that the video stream covers the entire wall. At the same time, the airborne rangefinder can be used to measure the distance between the lens and the wall. l , the unit is mm The laser radar sensor realizes the technology of simultaneous positioning and map building and combines the wheel encoder to obtain the moving speed. v , the unit is m / s .

[0019] Step S3, extracting the regional wall image sequence. Extracting the whole wall video image into a continuous regional wall image sequence. The regional cover image sequence is a determinant distribution. In order to extract the whole wall video image into a regional wall image sequence, the method for extracting the regional wall image sequence may specifically include: By using video coding and decoding technology, through video decompression, color space conversion, picture encoding and timestamp management, the key frame extraction interval is calculated by an adaptive key frame selection algorithm for the overall wall video image, and the regional wall image is extracted by rounding down the frame number interval to decompose the overall wall video image into a continuous sequence of regional wall images. The method for extracting regional wall images through key frames can include taking the video horizontally and the sensor size corresponding to the moving direction as a , the sensor size corresponding to the horizontal video shooting and moving direction is a, based on the imaging principle, the length of a single imaging is calculated as L=l×a / 1000×f , 60% pixel overlap is reserved for subsequent feature point matching, from which the frame interval is calculated to be t=0.4×n / v , the frame interval is rounded down and the image sequence is extracted. The same is true for other situations. This algorithm can effectively improve the efficiency of key frame extraction and reduce redundant calculations.

[0020] In order to ensure the clarity of the decomposed regional wall image and improve the accuracy of the trained instance segmentation neural network model in automatic detection of damage locations and operation information, the clarity quality of the extracted regional wall image can be filtered and screened. The blurriness of the regional wall image can be determined by the Laplace variance method, and the degree of overexposure or underexposure of the regional wall image can be determined by the histogram method. The regional wall images with poor quality such as blur, overexposure or underexposure can be removed. For the regional wall image with poor quality, the two regional wall images before and after can be extracted by the regional wall image extraction method, and the clarity filtering and judgment can be performed until the clarity requirements are met, and the clarity filtering and replacement is completed.

[0021] Step S4: preprocessing the wall image. Specifically, it may include: Step S41, image enhancement. For wall area images with uneven illumination and low contrast, histogram equalization and adaptive histogram equalization (CLAHE) technology are used to locally and adaptively enhance the contrast of the image and highlight the wall texture and damage features of the wall area image. Image enhancement can quickly identify the damage location and damage condition of the wall area image.

[0022] Step S42, noise reduction processing. Guided filtering and / or median filtering methods are used to perform graded noise reduction on the wall area image after image enhancement, and when removing the noise of the wall area image, its damage characteristics are retained. The noise reduction processing can improve the accuracy of identifying the damage location and damage condition of the wall area image.

[0023] Step S43, image correction. For distorted wall area images, feature points (such as corners and window frames) are identified in the wall area images, and the wall area images are corrected using perspective transformation and affine transformation algorithms, so that the wall appears as a regular rectangle in the wall area images. Image correction can facilitate subsequent splicing of the wall area images and improve the fusion accuracy of the following wall panoramic images. Step S4 is an optional step.

[0024] Step S5, fusion of the wall panoramic image. By extracting the feature points of the wall images of each area, matching the feature points of the wall images of adjacent areas, all the wall images of the areas are fused into a wall panoramic image covering the entire wall. The method for fusing the wall panoramic image may specifically include: Step S51, feature extraction: using the scale invariant feature transform (SIFT) and speeded up robust feature (SURF) algorithms to extract feature points and feature descriptions in the wall area image.

[0025] Step S52, feature matching: using a matching algorithm based on distance measurement (such as Euclidean distance, Hamming distance) to match the feature points of adjacent wall area images. For the matching results, the model parameters can be calculated and verified by the random sampling consensus (RANSAC) algorithm to remove mismatched points and improve matching accuracy.

[0026] Step S53, registration and fusion: according to the matched feature points, the wall area images are registered by using affine transformation and homography transformation methods, so that the same areas of adjacent wall area images are aligned in space; by calculating the transformation matrix, the wall area images are transformed into a unified coordinate system, and a higher fusion weight is given to the overlapping central part of the wall area images, and a lower fusion weight is given to the edge part, so as to form a complete wall panoramic image. This wall panoramic image fusion method can make the transition between the fused wall area images natural, avoid overlap, and improve the fusion quality and accuracy of the wall panoramic image.

[0027] Step S6, automatic detection of wall damage. Specifically, the wall panoramic image is adaptively segmented into a continuous sequence of wall block images according to the pixel size compressed by the neural network through the trained instance segmentation neural network model, and the wall damage situation of each wall block image is automatically detected and automatically labeled; all the wall block images after automatic detection and automatic labeling are spliced ​​to form a wall damage panoramic image covering the entire wall, and the wall damage location distribution of the entire wall of the existing building is displayed through the wall damage panoramic image, and the damage information of each wall damage location is displayed in detail. For example, if the size of the wall panoramic image is 6000×80000 pixels, and the compressed pixel size of the instance segmentation neural network model is 576×768, the wall panoramic image is cut into 10 rows×100 columns of images, and the trained instance segmentation neural network model is input respectively for automatic detection and automatic labeling of the damage situation, and the damage coordinate position, type, index (length, width, area) and other damage information of the damage are output to achieve quantitative evaluation of the damage. The adaptive cutting ratio can be adjusted according to the detection accuracy requirements.

[0028] The automatic detection and automatic labeling of the wall damage conditions of each wall block image refers to the automatic detection and automatic labeling of the damage location and damage information of the damaged wall block image, and no labeling of the wall block image without damage. The damage information includes but is not limited to wall pollution, cracking, defect, weathering, etc., and may also include damage shape outline, length, width, height, area, volume and other damage size information.

[0029] Please refer to Figure 3 The embodiment of the present invention further provides a system 100 for automatically detecting overall wall damage based on video stream, which adopts the above-mentioned method for automatically detecting overall wall damage based on video stream, and comprises: The model training module 110 is used to use the wall images that have been manually inspected for damage as sample data to perform automatic detection and automatic labeling training on the wall image damage through an instance segmentation neural network model.

[0030] The video dynamic acquisition module 120 is used to acquire the entire wall surface video image of a certain wall of the existing building in parallel with the wall surface of the existing building.

[0031] The regional wall image sequence extraction module 130 is used to extract the entire wall video image into a continuous regional wall image sequence.

[0032] The wall image preprocessing module 140 is used to perform image enhancement, noise reduction and image correction on the wall area image. The wall image preprocessing module 140 is an optional module.

[0033] The wall panoramic image fusion module 150 is used to extract feature points of the wall images of each area, match the feature points of the wall images of adjacent areas, and fuse all the wall images of the areas into a wall panoramic image covering the entire wall; The wall damage automatic detection module 160 is used to adaptively segment the wall panoramic image into a continuous wall block image sequence according to the pixel size compressed by the neural network through the trained instance segmentation neural network model, and automatically detect and automatically mark the wall damage of each wall block image; splice all the wall block images after automatic detection and automatic marking to form a wall damage panoramic image covering the entire wall surface, and use the wall damage panoramic image to provide an overall overview of the distribution of wall damage positions on the entire wall of the existing building and provide detailed annotations of the damage information of each wall damage position.

[0034] Please refer to Figure 4 The embodiment of the present invention also provides an automatic detection device 10 for overall wall damage based on video stream, which adopts the above-mentioned automatic detection system 100 for overall wall damage based on video stream. The automatic detection device 10 for overall wall damage based on video stream includes but is not limited to a high-definition camera (focal length f , sensor size a×b , frame rate n )'s drone 11, ground robot 12 or wall-climbing robot 13.

[0035] The embodiment of the present invention provides a method, system and device for automatic detection of overall wall damage based on video stream, which collects video images of the overall wall surface of a certain wall of an existing building wall, extracts regional wall image sequences from the collected video images, fuses the regional wall image sequences into a wall panoramic image, automatically segments the wall panoramic image into a wall block image sequence through a trained instance segmentation neural network model, and automatically detects and automatically labels the wall damage conditions, and splices all the wall block images after automatic detection and automatic labeling to form a wall covering the overall wall surface. The panoramic image of wall damage provides an overall overview of the distribution of wall damage positions on the entire wall of an existing building and displays detailed annotations of damage information on each wall damage position, thereby realizing automatic and intelligent detection of the overall wall damage of wide walls, vertical walls and other existing buildings that are continuous in a certain direction. It can achieve a dual display effect of global display of the distribution of damage positions on the entire wall of an existing building and detailed display of damage position nodes, so that the damage position and its damage information match, avoiding confusion caused by mismatch between the two. Compared with manual detection of damage conditions on wall entities and manual detection and annotation of damage conditions on a large number of wall area pictures, it has the advantages of high detection efficiency and high intelligence level, and can provide guidance for preventive protection of historical buildings.

[0036] The method, system and equipment for automatic detection of overall wall damage based on video stream provided in the embodiments of the present invention realize automatic inspection and automatic marking of overall wall damage of historical buildings through video stream, thereby improving the work efficiency and intelligence level in large-scale wall surface damage detection, and can be suitable for application scenarios of efficient damage inspection in daily maintenance of historical buildings.

[0037] The present invention is not limited to the above-mentioned specific implementation modes. Obviously, the above-mentioned embodiments are only some embodiments of the embodiments of the present invention, but not all embodiments. Based on the described embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention. Those skilled in the art can make other levels of modifications and changes to the present invention. In this way, if these modifications and changes of the present invention fall within the scope of the claims of the present invention, the present invention is also intended to include these changes and changes.

Claims

1. A method for automatic detection of overall wall damage based on video stream, characterized in that: include: Model training: Use the wall images that have been manually inspected for damage as sample data to train the automatic detection and automatic labeling of wall image damage through the instance segmentation neural network model; Video dynamic acquisition: parallel to the existing building wall, collect the entire wall video image of a certain wall of the existing building; Regional wall image sequence extraction: extract the whole wall video image into a continuous regional wall image sequence; Wall panoramic image fusion: Extract feature points of wall images in each area, match feature points of wall images in adjacent areas, and fuse all area wall images into a wall panoramic image covering the entire wall; Automatic detection of wall damage: The trained instance segmentation neural network model is used to adaptively segment the wall panoramic image into a continuous wall block image sequence according to the pixel size compressed by the neural network, and the wall damage of each wall block image is automatically detected and labeled; all the wall block images after automatic detection and automatic labeling are spliced ​​to form a wall damage panoramic image covering the entire wall surface. The wall damage panoramic image can be used to provide an overall overview of the distribution of wall damage locations on the entire wall of the existing building, and the damage information of each wall damage location is annotated and displayed in detail.

2. The method for automatic detection of overall wall damage based on video stream according to claim 1 is characterized in that: The instance segmentation neural network model is Mask R-CNN, YOLACT or an improved version of YOLO.

3. The method for automatic detection of overall wall damage based on video stream according to claim 1 is characterized in that: The method for extracting a regional wall image sequence comprises: Using video coding and decoding technology, through video decompression, color space conversion, image encoding and timestamp management, the key frame extraction interval of the overall wall video image is calculated by an adaptive key frame selection algorithm, and the regional wall image is extracted from the overall wall video image by rounding down the frame number interval, so as to decompose the overall wall video image into a continuous regional wall image sequence.

4. The method for automatic detection of overall wall damage based on video stream according to claim 1 is characterized in that: After the regional wall image sequence is extracted and before the wall panoramic image is fused, the method further includes: Wall image preprocessing: including: Image enhancement: For wall area images with uneven lighting and low contrast, histogram equalization and adaptive histogram equalization techniques are used to locally and adaptively enhance the image contrast and highlight the wall texture and damage features of the wall area images. Noise reduction processing: Guided filtering and / or median filtering methods are used to perform graded noise reduction on the wall area image after image enhancement, so as to retain its damage characteristics while removing the noise of the wall area image; Image correction: For distorted wall area images, feature points are identified in the wall area images and the wall area images are corrected using perspective transformation and affine transformation algorithms so that the wall appears as a regular rectangle in the wall area images.

5. The method for automatic detection of overall wall damage based on video stream according to claim 1 is characterized in that: The method for fusing the wall panoramic image comprises: Feature extraction: Scale-invariant feature transformation and accelerated robust feature algorithm are used to extract feature points and feature descriptions in the wall area image; Feature matching: A matching algorithm based on distance measurement is used to match the feature points of adjacent wall area images; Registration and fusion: Based on the matched feature points, the wall area images are registered using affine transformation and homography transformation methods to align the same areas of adjacent wall area images in space. By calculating the transformation matrix, the wall area images are transformed into a unified coordinate system, and a higher fusion weight is given to the overlapping central part of the wall area images, and a lower fusion weight is given to the edge part, so that the transition of the fused wall area images is natural, forming a complete panoramic image of the wall.

6. The method for automatic detection of overall wall damage based on video stream according to claim 5 is characterized in that: In the feature matching step: for the matching results, the model parameters are calculated and verified through a random sampling consistency algorithm, and the mismatching points are removed to improve the matching accuracy.

7. An automatic detection system for overall wall damage based on video stream, characterized in that: The method for automatically detecting overall wall damage based on video stream according to any one of claims 1 to 3 and 5 to 6 comprises: A model training module is used to use the wall images that have been manually inspected for damage as sample data to train the automatic detection and automatic labeling of the wall image damage through the instance segmentation neural network model; The video dynamic acquisition module is used to acquire the whole wall video image of a certain wall of an existing building in parallel with the wall of the existing building; A regional wall image sequence extraction module is used to extract the entire wall video image into a continuous regional wall image sequence; The wall panoramic image fusion module is used to extract the feature points of the wall images of each area, match the feature points of the wall images of adjacent areas, and fuse the wall images of all areas into a wall panoramic image covering the entire wall; The automatic wall damage detection module is used to adaptively segment the wall panoramic image into a continuous wall block image sequence according to the pixel size compressed by the neural network through the trained instance segmentation neural network model, and automatically detect and automatically mark the wall damage of each wall block image; all wall block images after automatic detection and automatic marking are spliced ​​to form a wall damage panoramic image covering the entire wall surface, and the wall damage panoramic image is used to provide an overall overview of the distribution of wall damage positions on the entire wall of the existing building and to provide detailed annotations of the damage information of each wall damage position.

8. The video stream-based automatic detection system for overall wall damage according to claim 7 is characterized in that: Also includes: The wall image preprocessing module is used to perform image enhancement, noise reduction and image correction on the wall area image.

9. An automatic detection device for overall wall damage based on video stream, characterized in that: The automatic detection system for overall wall damage based on video stream as described in claim 7 is adopted.

10. The automatic detection device for overall wall damage based on video stream according to claim 9 is characterized in that: The device is a drone, a ground robot or a wall-climbing robot equipped with a high-definition camera.

Images