A method for generating maintenance work orders based on spatial image recognition of building damage

Abstract

The present invention discloses a method for generating maintenance work orders based on spatial image recognition of building damage, including: obtaining orthographic panoramic color images of each surface of a building and correcting them into surface panoramic color images, and converting them into surface panoramic black-and-white images; respectively cropping and segmenting each panoramic color image and black-and-white image into regional grid color images and black-and-white images; drawing grid dividing lines on the regional grid black-and-white images to form regional number index images and cropping and segmenting them to form segmented black-and-white images as segmented annotation base maps; cropping and segmenting the regional grid color images into segmented color images, automatically detecting and annotating the damage situation through an instance segmentation algorithm and extracting the annotation layer as a segmented damage mask image; superimposing each segmented damage mask image onto the segmented annotation base map to form a segmented annotation damage map; extracting the damage information of the segmented annotation damage map to generate a damage form, and generating a maintenance work order from the regional number index image, the segmented annotation damage map and the damage form therein. The present invention has the advantages of high damage recognition efficiency and high accuracy.

Description

Technical Field

[0001] The present invention relates to the field of building maintenance construction, and particularly relates to a method for identifying building damage based on spatial images. Background Art

[0002] When conducting surveys on existing buildings, it is necessary to rely on historical drawings to verify and mark the current situation. However, many ancient buildings lack historical materials. Still, it is necessary for drafters to carefully measure the on-site dimensions and first draw standard technical drawings, and then mark the damaged parts obtained from the on-site survey on the two-dimensional technical drawings to guide on-site operations. The drawing operation process takes a long time and is prone to omissions. The positioning of the final generated damage annotation drawing is formed through two operations, resulting in a large error from the actual position, leading to inconsistent information during construction handover. In addition, the elevation damage situation is complex and the quantity is large. It is easy to be omitted only through manual identification, resulting in inaccurate statistics of the damaged area. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for generating a maintenance work order by identifying building damage based on spatial images, so as to solve the problems of low efficiency and low accuracy in manually surveying the damage situation of historical buildings and drawing engineering drawings with damage information.

[0004] To solve the above technical problems, the present invention provides a method for generating a maintenance work order by identifying building damage based on spatial images, including:

[0005] Step S1, obtaining orthographic panoramic color images of each surface of the building;

[0006] Step S2, measuring the external contour dimensions of the building to draw an external block diagram of the building, putting the orthographic combined color images of each surface into the corresponding frame of the external block diagram of the building, and performing distortion correction and edge cropping to form surface panoramic color images;

[0007] Step S3, respectively converting the surface panoramic color images of each surface into surface panoramic black-and-white images;

[0008] Step S4, cropping and dividing each surface panoramic color image and surface panoramic black-and-white image according to the same grid size and numbering them to form a number of regional grid color images and a number of regional grid black-and-white images with the same number and the same numbering;

[0009] Step S5, drawing grid dividing lines on the regional grid black-and-white images of each number according to the same grid size to form a regional number index map, cropping and dividing the regional grid black-and-white images of each number according to the drawn grid dividing lines and numbering them to form a number of numbered divided black-and-white images, and using the divided black-and-white images as divided annotation base maps;

[0010] Step S6: Cut and divide the area grid color maps with each number according to the specifications of the segmented annotation base map, and number them to form a number of segmented color maps that are the same in number and in one-to-one correspondence with the segmented annotation base map. Automatically detect and annotate the damage conditions of the segmented color maps with each number through the instance segmentation algorithm, and extract the annotation layer of the segmented color maps with each number after damage condition annotation as the segmented damage mask map;

[0011] Step S7: Superimpose the segmented damage mask maps with each number on the corresponding segmented annotation base maps with the same number to form the segmented annotation damage maps with this number;

[0012] Step S8: Extract the damage condition information of the segmented annotation damage maps with each number to generate a damage form, and generate a maintenance work order in a predetermined format for the area number index maps with each number, the segmented annotation damage maps with each number included therein, and the damage form, and export or print them.

[0013] Further, in the method for generating a maintenance work order based on spatial image recognition of building damage provided by the present invention, in step S8, the area number index maps with each number and the segmented annotation damage maps with each number are scaled in the same proportion.

[0014] Further, in the method for generating a maintenance work order based on spatial image recognition of building damage provided by the present invention, in step S6, the instance segmentation algorithm is the YOLACT algorithm.

[0015] Further, in the method for generating a maintenance work order based on spatial image recognition of building damage provided by the present invention, in step S5, the area grid black and white map is automatically cut and divided into segmented annotation base maps through the instance segmentation algorithm.

[0016] Further, in the method for generating a maintenance work order based on spatial image recognition of building damage provided by the present invention, in step S4, the panoramic color map of the surface and the panoramic black and white map of the surface are automatically cut and divided and numbered through the instance segmentation algorithm to form a number of area grid color maps and a number of area grid black and white maps with the same number and the same number.

[0017] Further, in the method for generating a maintenance work order based on spatial image recognition of building damage provided by the present invention, the order of step S5 and step S6 is interchanged.

[0018] Further, in the method for generating a maintenance work order based on spatial image recognition of building damage provided by the present invention, in step S3, the panoramic color map of the surface is decolorized and hue balanced to generate a panoramic black and white map of the surface.

[0019] Further, in the method for generating a maintenance work order based on spatial image recognition of building damage provided by the present invention, in step S2, the outer contour dimensions of the building are measured by a laser rangefinder or a total station.

[0020] Furthermore, in the method for generating maintenance work orders based on spatial image recognition of building damages provided by the present invention, in step S1, orthographic panoramic color images of each surface of the building are collected by a drone equipped with a camera.

[0021] Furthermore, in the method for generating maintenance work orders based on spatial image recognition of building damages provided by the present invention, photos of each surface of the building are taken by a drone equipped with a camera, and all the photos of each surface of the building are stitched together by a stitching synthesis algorithm to generate orthographic panoramic color images of each surface.

[0022] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0023] In the method for generating maintenance work orders based on spatial image recognition of building damages provided by the present invention, the orthographic panoramic color images of each surface of the obtained building are subjected to distortion correction and edge cropping according to the actual ratio of each surface of the building to form corrected surface panoramic color images, so that the corrected surface panoramic color images are consistent with the ratio of each surface of the building; the surface panoramic color images are converted into surface panoramic black-and-white images, and the surface panoramic color images and surface panoramic black-and-white images are cropped and segmented according to the same specification grid size and numbered to form regional grid color images and regional grid black-and-white images with the same numbers, so that the specification sizes and quantities of the regional grid color images and regional grid black-and-white images are consistent; a regional number index map is formed by drawing grid dividing lines on the regional grid black-and-white images of each number, and the regional grid black-and-white images of each number and the regional grid color images of each number are respectively cropped and divided into a number of block marked base maps and a number of block color images with the same quantity and corresponding numbers according to the same grid size; the damage conditions of the block color images of each number are automatically detected and marked by an instance segmentation algorithm, and the marked layer is extracted as a block damage mask image; the block damage mask images of each number are superimposed on the corresponding block marked base maps of the same number to form block marked damage images of the same number, so as to realize the automatic detection of the damage conditions of each surface of the historical building and generate block marked damage images that are easy to identify, avoiding overlooking in manual detection, eliminating the need to draw engineering drawings, and preventing the problem of incorrect marking of the survey positions on the engineering drawings, thereby improving the survey efficiency and accuracy of the damage conditions of historical buildings.

[0024] In the method for generating maintenance work orders based on spatial image recognition of building damages provided by the present invention, the damage condition information of the block marked damage images of each number is extracted and a damage form is generated, and the regional number index map of each number and the block marked damage images and damage forms of each number it contains are generated into maintenance work orders according to a predetermined format and exported or printed, so as to provide a reference basis for the maintenance, repair, and renovation construction of historical buildings through the maintenance work orders.

[0025] The method for generating maintenance work orders based on spatial image recognition of building damage provided by the present invention divides the panoramic color images and panoramic black-and-white images of the damaged surfaces of each side of the building into a number of regional grid color images and a number of regional grid black-and-white images of the same specification size, and then divides each regional grid color image and regional grid black-and-white image into a number of smaller-sized block color images and a number of block annotation base maps respectively for damage annotation processing and graphic overlay processing, avoiding the problems of difficult calculation and lag due to overly large graphics, improving the operation speed, and thus improving the survey efficiency.

[0026] The method for generating maintenance work orders based on spatial image recognition of building damage provided by the present invention forms a block annotation damage map of the corresponding number by extracting the block damage mask map of each numbered block color image after damage condition annotation and overlaying it on the block annotation base map of the corresponding number, so that the damage condition can be clearly, quickly, and intuitively observed on the block annotation base map, avoiding the problem that the annotation information color of the block color image for annotating the damage condition is the same as the color of the block color image and cannot be recognized and distinguished.

[0027] The method for generating maintenance work orders based on spatial image recognition of building damage provided by the present invention can quickly identify the damage conditions of the four elevations and one top surface of historical buildings for rapid survey, and can output a regional number index map and a block annotation damage map. The location of each block annotation damage map can be found in the regional number index map through the number of the block annotation damage map, which provides convenience for the maintenance of the building. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a flowchart of the method for generating maintenance work orders based on spatial image recognition of building damage;

[0029] Figure 2 is a regional number index map of a certain area of a certain side view of a building;

[0030] Figure 3 is a block annotation damage map of a certain number in a certain area of a certain side view of a building. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] The present invention will be described in detail below with reference to the accompanying drawings: According to the following description, the advantages and features of the present invention will be clearer. It should be noted that the accompanying drawings are all in a very simplified form and use non-precise scales, only for the purpose of facilitating and clearly assisting in explaining the purpose of the embodiments of the present invention.

[0032] Please refer to Figure 1 , the embodiment of the present invention provides a method for generating maintenance work orders based on spatial image recognition of building damage, including:

[0033] Step S1, obtain the orthographic projection panoramic color images of each face of the building. To quickly obtain the orthographic projection panoramic color images of each face of the building, a drone equipped with a camera can be used to collect the orthographic projection panoramic color images of each face of the building. Specifically: take photos of each face of the building with a drone equipped with a camera, and stitch all the photos of each face of the building through a stitching synthesis algorithm to generate the orthographic projection panoramic color images of each face.

[0034] Step S2, measure the external contour dimensions of the building to draw the external outline diagram of the building, place the orthographic projection combined color images of each face into the corresponding face frame of the external outline diagram of the building, and perform distortion correction and edge clipping to form the face panoramic color images. To draw the external outline diagram of the building with accurate dimensions, a laser rangefinder or total station can be used to measure the external contour dimensions of the building, and the external outline diagram of the building is drawn according to the measured dimension information.

[0035] Step S3, convert the face panoramic color images of each face into face panoramic black and white images respectively. To obtain the face panoramic black and white images, the face panoramic color images can be desaturated and hue balanced to generate the face panoramic black and white images.

[0036] Step S4, crop and divide the face panoramic color images and face panoramic black and white images of each face according to the same grid size and number them to form a number of regional grid color images and regional grid black and white images with the same number and the same numbering. To batch generate regional grid color images and regional grid black and white images quickly, an instance segmentation algorithm can be used to automatically crop and divide according to the size specifications of the drawn grid dividing lines to form a number of regional grid color images and regional grid black and white images with one-to-one correspondence and the same number.

[0037] Step S5, draw grid dividing lines on the regional grid black and white images of each number according to the same grid size to form a regional number index diagram. As Figure 2 shown, an instance segmentation algorithm can be used to automatically crop and divide the regional grid black and white images of each number according to the drawn grid dividing lines and number them to form a number of numbered block black and white images, and use the block black and white images as the block annotation base map. Among them Figure 2 an example of the drawing is an A3 drawing, with a scale of about 1:100 and a drawing length ratio of 4:3 for the regional number index diagram. The size specifications of the grid dividing lines include but are not limited to 3.6m × 2.4m. Figure 2 The numbers in

[0038] Step S6: Cut and divide the area grid color maps of each number according to the specifications of the segmented annotation base map and number them to form a number of segmented color maps with the same quantity as the segmented annotation base map and corresponding one-to-one numbers. Automatically detect and annotate the damage conditions of the segmented color maps of each number through the instance segmentation algorithm, and extract the annotation layer of the segmented color maps of each number with the damage conditions annotated as the segmented damage mask map. The order of Step S5 and Step S6 can be interchanged. The instance segmentation algorithm includes but is not limited to the YOLACT algorithm.

[0039] Step S7: Superimpose the segmented damage mask maps of each number on the corresponding segmented annotation base map of the same number to form the segmented annotation damage map of this number, as Figure 3 shown. The damage information of the segmented annotation damage map includes but is not limited to drawing number, legend, damage location, damage type, damage contour, and damage area. The damage type includes but is not limited to cracking, epiphyte, cracking, defect, efflorescence, etc. Among them Figure 3 illustrates the defect situation.

[0040] Step S8: Extract the damage condition information of the segmented annotation damage maps of each number to generate a damage form, and generate a maintenance work order in a predetermined format from the area number index maps of each number, the segmented annotation damage maps of each number included therein, and the damage form for export or printing. To facilitate reading the maintenance work order, scale the area number index maps of each number and the segmented annotation damage maps of each number in the same proportion. When Figure 2 exporting the area number index map, after scaling the drawing at a ratio of 1:100, it can be printed on the drawing sheet of A3 or A2 format according to the actual size of a certain side of the building. Among them Figure 3 The export or printing ratio illustrated is 1:10. After testing, the complete damage form can be presented on the A3 drawing sheet. The size of each damage can be viewed through the corresponding form, which is convenient for quickly locating the damage position and can be used to count the overall number of damages and compare the damage severity of different parts, assisting in the rapid formulation of on-site construction plans.

[0041] The method for generating maintenance work orders based on spatial image recognition of building damage provided by the embodiments of the present invention corrects the distortion and trims the edges of the panoramic color images of each face of the building obtained according to the actual ratio of each face of the building to form corrected face panoramic color images, so that the corrected face panoramic color images are consistent with the ratio of each face of the building; converts each face panoramic color image into a face panoramic black-and-white image, trims and divides each face panoramic color image and the face panoramic black-and-white image according to the same specification grid size and numbers them to form area grid color images and area grid black-and-white images with the same numbers, so that the specification sizes of the area grid color images and the area grid black-and-white images are consistent; draws grid dividing lines on the area grid black-and-white images of each number to form area number index images, and trims and divides the area grid black-and-white images of each number and the area grid color images of each number into a number of block annotation base maps and a number of block color images with the same number and corresponding one-to-one according to the same grid size; automatically detects and annotates the damage conditions of the block color images of each number through an instance segmentation algorithm and extracts the annotation layer as a block damage mask image; superimposes the block damage mask images of each number on the corresponding block annotation base maps of the same number to form block annotation damage images of the same number, thereby realizing the automatic detection of the damage conditions of each face of the historical building and generating block annotation damage images that are convenient to identify, avoiding omissions in manual detection, eliminating the need to draw engineering drawings, and preventing the problem of confusion in the survey location markings on the engineering drawings, thus improving the survey efficiency and accuracy of the damage conditions of historical buildings.

[0042] The method for generating maintenance work orders based on spatial image recognition of building damage provided by the embodiments of the present invention extracts the damage condition information of the block annotation damage images of each number and generates a damage form, and generates, exports or prints a maintenance work order in a predetermined format according to the area number index images of each number and all the block annotation damage images and the damage form they contain, thereby providing a reference basis for the maintenance and repair construction of historical buildings through the maintenance work order.

[0043] The method for generating maintenance work orders based on spatial image recognition of building damage provided by the embodiments of the present invention divides the face panoramic color images and the face panoramic black-and-white images with damage information on each face of the building into a number of area grid color images and a number of area grid black-and-white images with the same specification size, and then divides each area grid color image and area grid black-and-white image into a number of smaller block color images and a number of block annotation base maps with smaller specification sizes for damage annotation processing and graphic overlay processing respectively, avoiding the problems of too large graphics, difficult calculation and freezing, improving the operation speed, and thus improving the survey efficiency.

[0044] The method for generating a maintenance work order based on spatial image recognition of building damage provided by the embodiments of the present invention extracts the block damage mask images of the block color images with damage conditions marked by each number and superimposes them on the block annotation base map with the corresponding number to form the block annotation damage map of this number, so that the damage conditions can be clearly, quickly and intuitively observed on the block annotation base map, avoiding the problem that the colors of the annotation information of the block color images with marked damage conditions are the same as those of the block color images and cannot be recognized and distinguished.

[0045] The method for generating a maintenance work order based on spatial image recognition of building damage provided by the embodiments of the present invention can quickly identify the damage conditions of the four facades and one top surface of a historical building for rapid survey, and can output a regional number index map and a block annotation damage map. The location of each block annotation damage map can be found in the regional number index map through the number of the block annotation damage map, which provides convenience for the maintenance of the building.

[0046] The method for generating a maintenance work order based on spatial image recognition of building damage provided by the embodiments of the present invention is particularly applicable to the situation where there are no drawings and BIM models of historical buildings.

[0047] The present invention is not limited to the above specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the described embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present invention. Those skilled in the art can make other levels of modifications and changes to the present invention. Thus, if these modifications and changes of the present invention are within the scope of the claims of the present invention, the present invention also intends to include these modifications and changes.

Claims

1. A method for generating maintenance work orders based on spatial image recognition of building damage, characterized in that, Including: Step S1: Obtain the orthographic projection panoramic color images of each side of the building; Step S2: Measure the outer contour dimensions of the building to draw the outer frame diagram of the building. Place the orthographic projection composite color images of each side into the corresponding frame of the outer frame diagram of the building, and perform distortion correction and edge clipping to form the corrected panoramic color images of the sides; Step S3: Convert the panoramic color images of each side into panoramic black-and-white images of the sides; Step S4: Crop and divide the panoramic color images and panoramic black-and-white images of each side according to the same grid size and number them to form a number of regional grid color images and regional grid black-and-white images with the same number and the same numbering; Step S5: Draw grid dividing lines on the regional grid black-and-white images of each number according to the same grid size to form a regional number index diagram. Crop and divide the regional grid black-and-white images of each number according to the drawn grid dividing lines and number them to form a number of divided black-and-white images with numbers, and use the divided black-and-white images as the divided annotation base maps; A number of divided color images formed by cropping, dividing and numbering the regional grid color images of each number according to the same grid size. Automatically detect and label the damage conditions of the divided color images of each number through the instance segmentation algorithm. Extract the annotation layer of the divided color images of each number after labeling the damage conditions as the divided damage mask images. The number of divided annotation base maps and divided color images is the same, the specification sizes are the same, and the numbers correspond one by one; Step S6: Overlay the divided damage mask images of each number on the corresponding divided annotation base maps of the same number to form the divided annotation damage maps of this number; Step S7: Extract the damage condition information of the divided annotation damage maps of each number to generate a damage form. Generate a maintenance work order in a predetermined format from the regional number index diagrams of each number and the divided annotation damage maps and damage forms included therein, and export or print it.

2. The method for generating a maintenance work order based on spatial image recognition of building damage according to claim 1, wherein, In step S7, scale the regional number index diagrams of each number and the divided annotation damage maps included therein by the same ratio.

3. The method for generating a maintenance work order based on spatial image recognition of building damage according to claim 1, characterized in that, In step S5, the instance segmentation algorithm is the YOLACT algorithm.

4. The method for generating a maintenance work order based on spatial image recognition of building damage according to claim 1, characterized in that, In step S5, automatically crop and divide the regional grid black-and-white images into divided annotation base maps through the instance segmentation algorithm.

5. The method for generating a maintenance work order based on spatial image recognition of building damage according to claim 1, wherein In step S4, automatically crop, divide and number the panoramic color images and panoramic black-and-white images of each side through the instance segmentation algorithm to form a number of regional grid color images and regional grid black-and-white images with the same number and the same numbering.

6. The method for generating a maintenance work order based on spatial image recognition of building damage according to claim 1, characterized in that, In step S3, perform color removal processing and hue balance processing on the panoramic color images of the sides to generate panoramic black-and-white images of the sides.

7. The method for generating a maintenance work order based on spatial image recognition of building damage according to claim 1, wherein, In step S2, measure the outer contour dimensions of the building by a laser rangefinder or total station.

8. The method for generating a maintenance work order based on spatial image recognition of building damage according to claim 1, characterized in that, In step S1, collect the orthographic projection panoramic color images of each side of the building by a drone equipped with a camera.

9. The method for generating a maintenance work order based on spatial image recognition of building damage according to claim 8, characterized in that, Take photos of each side of the building by a drone equipped with a camera, and splice all the photos of each side of the building through a splicing synthesis algorithm to generate the orthographic projection panoramic color images of each side.

Images