A fault locating method and system for spliced image floor defect detection

Abstract

The application provides a fault positioning method and corresponding system for spliced image floor defect detection, mainly including window segmentation, image splicing, window positioning, defect positioning, etc. The method firstly splices the collected floor image sequence, extracts features and geometrically corrects the floor area, obtains the corresponding relationship of pixels in the reference floor coordinate system in the photographed image, and accurately positions the defects output by the defect automatic identification algorithm. The method can effectively overcome the problem that the order and position of the same floor area in the picture sequence are different due to the uncertainty of the lens position and angle in each shooting, can significantly improve the defect positioning accuracy in the floor defect detection process using spliced images, has high positioning result reliability, and is suitable for popularization and application in floor defect detection and maintenance work.

Description

Technical Field

[0001] This invention belongs to the field of defect detection technology, and specifically relates to a fault location method and system for detecting defects in spliced ​​images of building floors. Background Technology

[0002] Over time, exposure to sunlight, wind, rain, and snow can cause decorative bricks to fall off, break, or even crack on building exteriors. Photographic methods offer advantages in detecting these defects, being accurate, economical, and efficient. However, in practice, it's often necessary to photograph the building from a considerable distance (greater than 50 meters). To reduce geometric distortion and improve image resolution, telephoto lenses of 200mm or more are required. Due to the limited field of view of telephoto lenses, each image only covers a localized area of ​​the building. Furthermore, the uncertainty of lens position and angle during each shot leads to variations in the location and order of the same defect within the image sequence, complicating defect localization. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention provides a fault location method and system for detecting floor defects using stitched images.

[0004] The specific technical solution of this invention is as follows:

[0005] This invention provides a fault location method for detecting floor defects in stitched images, comprising the following steps:

[0006] A series of original images covering the entire floor area are acquired as the original image sequence. The windows in each of the original images are segmented using a trained convolutional neural network to obtain a binary mask image with the same size as the original image. All the mask images are then sorted to obtain a mask sequence.

[0007] The adjacent original images are sequentially subjected to feature extraction, matching, and image stitching to obtain a global image; the same method is used to extract features, match, and stitch the mask image to obtain a global mask; the location of the defect is recalculated and located according to the projection transformation;

[0008] Connectivity detection is performed on the global mask to locate all windows on the floor and determine the door number corresponding to each window;

[0009] The location of the defect after repositioning is found from the global mask, and the nearest window is located. Based on this, its position in the overall floor plan is calculated, thus completing the positioning.

[0010] Furthermore, the specific method for feature extraction is as follows:

[0011] a. Use Gaussian blur with different parameters to represent different scale spaces, detect feature points that exist at different scales, and thus detect extreme points;

[0012] b. Remove unstable extrema, including low-contrast extrema and unstable edge response points;

[0013] c. Using the feature point as the center, calculate the angle and magnitude of the gradient of each pixel in its neighborhood, use a histogram to statistically analyze the angle of the gradient, and determine the direction of the feature point;

[0014] d. Rotate the coordinate axes to align with the direction of the feature point, centered on the feature point. The gradient magnitude and direction of the pixels in the window are used to divide the pixels in the window into 16 blocks. The histograms of the eight directions in each block are statistically analyzed to form a 128-dimensional feature vector.

[0015] Furthermore, the specific methods for matching and image stitching are as follows:

[0016] a. For the first original image in the original image sequence or the first mask image in the mask sequence, use a KD-tree data structure to store the feature description vector in an orderly manner, and then use a fast approximate k-nearest neighbor algorithm to find the nearest neighbor and the second nearest neighbor.

[0017] b. Calculate the parameters of the perspective transformation matrix based on the feature points; transform the next original image or the mask image using the perspective transformation function, and use a polynomial interpolation algorithm to estimate the non-integer coordinate pixel values ​​to obtain the calibrated image;

[0018] c. Stitch the previous original image with the transformed next original image, or stitch the previous mask image with the transformed next mask image. The overlapping part can be weighted averaged to obtain the stitched image, and then continue to stitch it with the next original image or mask image.

[0019] Furthermore, the specific method for recalculating and locating the defect is as follows:

[0020] Let the original center position of the defect be (xbreak, ybreak). According to the transmission transformation matrix...

[0021]

[0022] The coordinate position of the defect after image stitching can be calculated. as follows:

[0023] .

[0024] Furthermore, the specific method for locating all windows on the floor is as follows:

[0025] a. Initialization Store C0 into Group0, set the mean x-coordinate of the center of the connected components in Group0 to mean(Group0), and set the number of elements in Gp to numG=1;

[0026] b. Traverse all connected components and perform the following operation on each connected component Ci: (e.g.) If numG = numG + 1, then Ci is placed in Group n; if numG = numG + 1, then Ci is placed in Group numG.

[0027] Furthermore, the method for locating all windows on the floor also includes the following steps:

[0028] c. Traverse Groupn in Gp, sort the connected components in Groupn according to the y-coordinate from smallest to largest to obtain SortGroupn, and calculate the minimum y-coordinate spacing Dn;

[0029] d. Iterate sequentially through each SortGroupn, and let the difference between the y-coordinates of the centers of the k-th and (k+1)-th connected components be y = kx. If dk / Dn > 1, it means there is an undivided window between the two windows. Connect the two windows according to... Divide the data into equal parts, and also include the division points as window positions in SortGroupn;

[0030] e. Sort the new SortGroupn in ascending order of x-coordinate mean to form a set SortGp, and calculate the area of ​​the connected components within the group based on the average area of ​​the connected components. Select the components with an area greater than a preset threshold as the main window.

[0031] Furthermore, the specific method for locating the defect within the overall floor plan is as follows:

[0032] Calculate the coordinates after image stitching. Find the nearest window by calculating the distance from the center of all windows in SortGp. Determine the horizontal position of the window based on n in SortGroupn in SortGp, and the vertical position based on k in SortGroupn, thus completing the positioning.

[0033] Another aspect of the present invention provides a fault location system for detecting floor defects using stitched images and operating the above-described method, comprising a window segmentation module, an image stitching module, a window location module, and a defect location module connected in sequence, wherein the window segmentation module is configured as follows:

[0034] A series of original images covering the entire floor area are acquired as the original image sequence. The windows in each of the original images are segmented using a trained convolutional neural network to obtain a binary mask image with the same size as the original image. All the mask images are then sorted to obtain a mask sequence.

[0035] The image stitching module is configured as follows:

[0036] The adjacent original images are sequentially subjected to feature extraction, matching, and image stitching to obtain a global image; the same method is used to extract features, match, and stitch the mask image to obtain a global mask; the location of the defect is recalculated and located according to the projection transformation;

[0037] The window positioning module is configured as follows:

[0038] Connectivity detection is performed on the global mask to locate all windows on the floor and determine the door number corresponding to each window;

[0039] The defect location module is configured as follows:

[0040] The location of the defect after repositioning is found from the global mask, and the nearest window is located. Based on this, its position in the overall floor plan is calculated, thus completing the positioning.

[0041] Furthermore, the image stitching module includes a feature extraction unit, a matching stitching unit, and a relocation unit, wherein the feature extraction unit is configured as follows:

[0042] Gaussian blur with different parameters is used to represent different scale spaces to detect feature points that exist at different scales, thereby detecting extrema. Unstable extrema, including low-contrast extrema and unstable edge response points, are removed. The gradient argument and magnitude of each pixel in the neighborhood of the feature point are calculated, and a histogram is used to statistically analyze the gradient argument to determine the direction of the feature point. The coordinate axes are rotated to reflect the direction of the feature point, centered on the feature point. The gradient magnitude and direction of the pixels in the window are used to divide the pixels in the window into 16 blocks. The histograms of the eight directions in each block are used to form a total of 128-dimensional feature vectors.

[0043] The matching and splicing unit is configured as follows:

[0044] For the first original image in the original image sequence or the first mask image in the mask sequence, a KD-tree data structure is used to store the feature description vector in an orderly manner, and then a fast approximate k-nearest neighbor algorithm is used to find the nearest and second nearest neighbors. The parameters of the perspective transformation matrix are calculated based on the feature points. The perspective transformation function is used to transform the next original image or the mask image, and a polynomial interpolation algorithm is used to estimate the non-integer coordinate pixel values ​​to obtain the calibrated image. The previous original image is then stitched together with the transformed next original image, or the previous mask image is stitched together with the transformed next mask image. The overlapping part is weighted and averaged to obtain the stitched image, and then stitched together with the next original image or mask image.

[0045] The defect relocation unit is configured as follows:

[0046] Let the original center position of the defect be (xbreak, ybreak). According to the transmission transformation matrix...

[0047]

[0048] The coordinates of the defect after image stitching can be calculated. as follows:

[0049] .

[0050] Furthermore, the preliminary positioning unit is configured as follows:

[0051] initialization Store C0 into Group0. Set the mean x-coordinate of the center of each connected component in Group0 to mean(Group0), and set the number of elements in Gp to numG=1. Iterate through all connected components and perform the following operations on each connected component Ci: If numG = numG + 1, then Ci is placed in Group n; if numG = numG + 1, then Ci is placed in Group numG.

[0052] The speculative completion unit is configured as follows:

[0053] Traverse Groupn in Gp, sort the connected components in Groupn according to the y-coordinate from smallest to largest to obtain SortGroupn, and calculate the minimum y-coordinate spacing Dn;

[0054] For each SortGroupn, iterate sequentially, and let the difference between the y-coordinates of the centers of the k-th and (k+1)-th connected components be y = kx. If dk / Dn > 1, it means there is an undivided window between the two windows. Connect the two windows according to... Divide the data into equal parts, and place the division points as window positions into SortGroupn; sort the new SortGroupn by the mean of the x-coordinate from smallest to largest to form a set SortGp, and calculate based on the average area of ​​the connected components within the group, and determine the main window based on the area greater than a preset threshold.

[0055] At this time, the defect location module is configured as follows:

[0056] Calculate the coordinates after image stitching. Find the nearest window by calculating the distance from the center of all windows in SortGp. Determine the horizontal position of the window based on n in SortGroupn in SortGp, and the vertical position based on k in SortGroupn, thus completing the positioning.

[0057] The beneficial effects of this invention are as follows: This invention provides a fault location method and corresponding system for floor defect detection using stitched images. It mainly includes window segmentation, image stitching, window location, and defect location. The method first stitches together the acquired floor image sequence and performs feature extraction and geometric correction on the floor area to obtain the correspondence between pixels in the captured images and the reference floor coordinate system, thereby accurately locating the defects output by the automatic defect recognition algorithm. This method effectively overcomes the problem of differences in the order and position of the same area on the floor in the image sequence caused by the uncertainty of the lens position and angle during each shot. In the process of floor defect detection using stitched images, it can significantly improve the accuracy of defect location, and the location results have high reliability, making it suitable for widespread application in floor defect detection and maintenance. Attached Figure Description

[0058] Figure 1 This is a flowchart of the fault location method for detecting floor defects in stitched images as described in Example 1;

[0059] Figure 2 This is a schematic diagram of the fault location system for detecting floor defects using stitched images as described in Example 2;

[0060] Figure 3 This is a sequence of original floor images collected in the application example;

[0061] Figure 4 This is a schematic diagram of obtaining a mask image from the original image in an application example, where (a) is the original image of the floor and (b) is its mask image;

[0062] Figure 5 This is the stitched global image in the application example;

[0063] Figure 6 This is the global mask stitched together in the application example;

[0064] Figure 7 This is a schematic diagram illustrating the positioning of the window center in an application example;

[0065] Figure 8 A diagram illustrating window completion and categorization in the application example;

[0066] Figure 9 This is a schematic diagram of floor defect location in an application example. Detailed Implementation

[0067] The present invention will be further described in detail below with reference to the accompanying drawings and the following embodiments.

[0068] Example 1

[0069] like Figure 1 As shown, this embodiment 1 provides a fault location method for detecting floor defects in stitched images, including the following steps:

[0070] S1: Window Segmentation: First, the trained Mask R-CNN is used to segment the windows in each original image I in the original image sequence, resulting in a binarized mask image I with the same size as the original image. mask Where 0 represents a window and 255 represents a non-window; according to the original image sequence {I mask The mask sequence is obtained by sorting the original images in order and then sorting the mask images.

[0071] S2: Image stitching: Since adjacent photos are also spatially adjacent during shooting, feature points are found in all original images of the original image sequence and stitched together to obtain the global image G. The same stitching process is then performed on all mask images in the mask sequence to obtain the global mask G. mask Simultaneously, the defect location is repositioned, and the coordinates of the original center position of the defect are obtained after image stitching. ;

[0072] S3: Window positioning: First, the stitched global mask image G... mask Perform connected component detection, setting the connectivity mode to 8-adjacency, and calculate the i-th connected component C. i The center is The area of ​​the connected region is A i Based on the x-coordinate of the center of the connected component i For connected component C i Perform clustering, grouping k belonging to a column n The windows are divided into groups, resulting in N sets of index sequences. ;

[0073] S4: Defect Location: Calculate the coordinates after image stitching. By comparing the distance to the center of all windows in the index sequence, the nearest window is found, and its position in the overall floor plan is calculated, thus completing the positioning.

[0074] In some specific embodiments, the specific method for extracting floor image features with rotation, scaling, and translation invariance from the original image or mask image using the SIFT operator is as follows:

[0075] a. Extremum detection in Differential Gaussian (DoG) scale space: Using Gaussian blur with different parameters to represent different scale spaces, feature points that exist at different scales are detected, thereby detecting extremum points;

[0076] b. Remove unstable extrema, mainly low-contrast extrema and unstable edge response points;

[0077] c. Determine the direction of the feature point: Taking the feature point as the center, calculate the argument and magnitude of the gradient of each pixel in its neighborhood. Then, use a histogram to count the argument of the gradient. The horizontal axis of the histogram is the direction of the gradient, and the vertical axis is the cumulative value of the gradient magnitude corresponding to the gradient direction. The direction corresponding to the highest peak in the histogram is the direction of the feature point.

[0078] d. Generate descriptors for feature points: First, rotate the coordinate axes to align with the direction of the feature points, centered on the feature points. The gradient magnitude and direction of the pixels in the window are used to divide the pixels in the window into 16 blocks. The histograms of the eight directions in each block are used to count the gradient magnitude and direction of the pixels, which can form a 128-dimensional feature vector.

[0079] In some specific embodiments, the specific methods for stitching together the original image or the mask image are as follows:

[0080] a. Perform SIFT feature point matching: First, for the first image (original image or mask image), use the KD tree (KDimensional Tree) data structure to store the feature description vector in an orderly manner, and then use the fast approximate k nearest neighbor (FLANN) algorithm to find the nearest neighbor and the second nearest neighbor;

[0081] b. Take 4 sets of SIFT transform feature points and calculate the parameters of the perspective transformation matrix; use the perspective transformation function to transform the next image (the next original image or mask image arranged in sequence), and use a polynomial interpolation algorithm to estimate the non-integer coordinate pixel values ​​to obtain the calibrated image;

[0082] c. Stitch the previous image with the transformed next image, using a weighted average for the overlapping portion. The resulting stitched image is then used to stitch with the next image. Simultaneously, the same method is applied to two adjacent I images. mask The images are stitched together to obtain a stitched mask image, which is then used for subsequent mask stitching.

[0083] In some specific embodiments, the specific methods for relocating defects are as follows:

[0084] Let the original center position of the defect be (xbreak, ybreak). According to the transmission transformation matrix...

[0085]

[0086] The coordinates of the defect after image stitching can be calculated. as follows:

[0087] .

[0088] In some specific embodiments, the clustering algorithm for window localization is as follows:

[0089] (1) Initialization Store C0 into Group0. Let the mean x-coordinate of the center of the connected components in Group0 be mean(Group0), and let G be... p The number of elements in the middle (num) G =1;

[0090] (2) Traverse all connected components, and for each connected component C i Perform the following operations: (If t is set to 10), then C i Add to Group n In the middle; such as num G =num G +1, then put Ci into Group. numG middle.

[0091] In some specific embodiments, since there may be missed detections during the window segmentation masking process, it is necessary to rely on existing G... p The locations of other possible windows are inferred and filled in. The specific algorithm for filling in the window is as follows:

[0092] (1) For G p Group in n Perform traversal and group n The connected components in the array are sorted in ascending order of their y-coordinates to obtain the SortGroup. n And calculate the minimum ordinate spacing D.n ;

[0093] (2) For each SortGroup n Perform a sequential traversal, and let the difference between the y-coordinates of the centers of the k-th and (k+1)-th connected components be y = k / k. If d k / D n If the value is greater than 1, it indicates that there is an undivided window between the two windows. Connect the two windows according to... Divide into equal parts, and add the division points as window positions to the SortGroup. n middle;

[0094] (3) Create a new SortGroup n Sort the data by sorting it in ascending order of the mean of the x-coordinate to form a set SortG. p The calculation is based on the average area of ​​the connected components within the group, and those with an area greater than 600 are considered as the main window (balcony).

[0095] Based on the preliminary estimation and completion of the window, the defect location can be carried out using the following method:

[0096] Calculate the coordinates after image stitching. With SortG p Find the nearest window by tracing the distance from the center of all windows in the SortG array. p SortGroup n n is used to determine the horizontal position (cell number) and is then assigned to the SortGroup. n The index k in the table is used to deduce its vertical position (based on the total number of floors, the floor it is on is determined from top to bottom), thus completing the location.

[0097] Example 2

[0098] like Figure 2 As shown, this embodiment 2 provides a fault location system for detecting floor defects using stitched images based on the method provided in embodiment 1. The system includes a window segmentation module 1, an image stitching module 2, a window location module 3, and a defect location module 4 connected in sequence. The window segmentation module 1 is configured as follows:

[0099] A series of original images covering the entire floor area are acquired as the original image sequence. The windows in each original image are segmented using a trained convolutional neural network to obtain a binary mask image with the same size as the original image. All mask images are then sorted to obtain a mask sequence.

[0100] Image stitching module 2 is configured as follows:

[0101] The adjacent original images are sequentially subjected to feature extraction, matching, and image stitching to obtain a global image; the same method is used to extract features, match, and stitch the mask image to obtain a global mask; the location of the defect is recalculated and located according to the projection transformation;

[0102] Window positioning module 3 is configured as follows:

[0103] Perform connected component detection on the global mask, locate all windows on the floor, and determine the door number corresponding to each window;

[0104] Defect location module 4 is configured as follows:

[0105] The location of the defect after repositioning is found from the global mask, and the nearest window is located. Based on this, its position in the overall floor plan is calculated, thus completing the positioning.

[0106] In some specific embodiments, the image stitching module 2 includes a feature extraction unit 21, a matching stitching unit 22, and a relocation unit 23, wherein the feature extraction unit 21 is configured as follows:

[0107] Gaussian blur with different parameters is used to represent different scale spaces to detect feature points that exist at different scales, thereby detecting extreme points; unstable extreme points are removed, including low-contrast extreme points and unstable edge response points; the gradient argument and magnitude of each pixel in the neighborhood of the feature point are calculated, and a histogram is used to statistically analyze the gradient argument. The horizontal axis of the histogram is the gradient direction, and the vertical axis is the cumulative value of the gradient magnitude corresponding to the gradient direction. The direction corresponding to the highest peak in the histogram is the direction of the feature point; the coordinate axes are rotated to the direction of the feature point, and the coordinates are then rotated to the feature point. The gradient magnitude and direction of the pixels in the window are used to divide the pixels in the window into 16 blocks. The histograms of the eight directions in each block are used to form a total of 128-dimensional feature vectors.

[0108] Matching splicing unit 22 is configured as follows:

[0109] For the first original image in the original image sequence or the first mask image in the mask sequence, a KD-tree data structure is used to store the feature description vector in an orderly manner. Then, the fast approximate k-nearest neighbor algorithm is used to find the nearest and second nearest neighbors. The parameters of the perspective transformation matrix are calculated based on the feature points. The perspective transformation function is used to transform the next original image or mask image, and a polynomial interpolation algorithm is used to estimate the non-integer coordinate pixel values ​​to obtain the calibrated image. The previous original image is then stitched together with the transformed next original image, or the previous mask image is stitched together with the transformed next mask image. The overlapping part is weighted and averaged to obtain the stitched image, which is then stitched together with the next original image or mask image.

[0110] Defect relocation unit 23 is configured as follows:

[0111] Let the original center position of the defect be (xbreak, ybreak). According to the transmission transformation matrix...

[0112]

[0113] The coordinates of the defect after image stitching can be calculated. as follows:

[0114] .

[0115] In some specific embodiments, the window positioning module 3 includes a preliminary positioning unit 31 and a prediction and completion unit 32. The preliminary positioning unit 31 is configured as follows:

[0116] initialization Store C0 into Group0. Set the mean x-coordinate of the center of each connected component in Group0 to mean(Group0), and set the number of elements in Gp to numG=1. Iterate through all connected components and perform the following operations on each connected component Ci: (If t is set to 10), then Ci is placed in Group n; if numG = numG + 1, then Ci is placed in Group numG.

[0117] Speculative completion unit 32 is configured as follows:

[0118] Traverse Groupn in Gp, sort the connected components in Groupn according to the y-coordinate from smallest to largest to obtain SortGroupn, and calculate the minimum y-coordinate spacing Dn;

[0119] For each SortGroupn, iterate sequentially, and let the difference between the y-coordinates of the centers of the k-th and (k+1)-th connected components be y = kx. If dk / Dn > 1, it means there is an undivided window between the two windows. Connect the two windows according to... Divide the data into equal parts, and place the division points as window positions into SortGroupn; sort the new SortGroupn by the mean of the x-coordinate from smallest to largest to form a set SortGp, and calculate based on the average area of ​​the connected components within the group, and determine the main window if the area is greater than the preset threshold (600).

[0120] At this time, defect location module 4 is configured as follows:

[0121] Calculate the coordinates after image stitching. With SortG p Find the nearest window by tracing the distance from the center of all windows in the SortG array. p SortGroup n n is used to determine the horizontal position (cell number) and is then assigned to the SortGroup. n The index k in the table is used to deduce its vertical position (based on the total number of floors, the floor it is on is determined from top to bottom), thus completing the location.

[0122] Application Examples

[0123] We collected real-life images of walls in a residential community, with a resolution of 7952*5304. Each file was approximately 30MB in size. We then created an image like this... Figure 3 The original image sequence shown.

[0124] Based on S1, Mask R-CNN is used to perform window detection on the original images to obtain the window mask corresponding to each original image, such as... Figure 4 As shown; according to S2, the original image sequence and the mask sequence are spliced ​​together to obtain the spliced ​​global original image (as shown). Figure 5 (as shown) and global mask (as shown) Figure 6 As shown), and reposition the defect location; according to S3, locate the window center (as shown). Figure 7 (as shown), and complete the list of undetected windows and classify the main windows (such as...). Figure 8 As shown in the figure, columns B, D, G, J, L, and N are the main form; finally, according to S4, the floor and unit numbers of the transformed defects are located and labeled, as shown in the figure. Figure 9 As shown, the two defects are located at "Unit 4, Floor 21" and "Unit 6, Floor 13" respectively. This allows for the rapid and accurate location of floor defects.

[0125] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A fault location method for detecting floor defects using stitched images, characterized in that, Includes the following steps: A series of original images covering the entire floor area are acquired as the original image sequence. The windows in each of the original images are segmented using a trained convolutional neural network to obtain a binary mask image with the same size as the original image. All the mask images are then sorted to obtain a mask sequence. The adjacent original images are sequentially subjected to feature extraction, matching, and image stitching to obtain a global image; the same method is used to extract features, match, and stitch the mask image to obtain a global mask. The location of the defect is recalculated and repositioned based on the transmission transformation. Connectivity detection is performed on the global mask to locate all windows on the floor and determine the door number corresponding to each window; The location of the defect after repositioning is found from the global mask, and the window closest to the repositioned location of the defect is found. Based on this, the location of the defect in the overall floor plan is calculated. The location in the overall floor plan includes the unit and floor number, thereby completing the positioning. The specific method for recalculating and locating the defect is as follows: Let the original center position of the defect be (xbreak, ybreak). According to the transmission transformation... The coordinates of the defect were calculated after image stitching. as follows: 。 2. The fault location method for detecting floor defects using stitched images as described in claim 1, characterized in that, The specific method for feature extraction is as follows: a. Use Gaussian blur with different parameters to represent different scale spaces, detect feature points that exist at different scales, and thus detect extreme points; b. Remove unstable extrema, including low-contrast extrema and unstable edge response points; c. Using the feature point as the center, calculate the angle and magnitude of the gradient of each pixel in its neighborhood, use a histogram to statistically analyze the angle of the gradient, and determine the direction of the feature point; d. Rotate the coordinate axes to align with the direction of the feature point, centered on the feature point. The gradient magnitude and direction of the pixels in the window are used to divide the pixels in the window into 16 blocks. The histograms of the eight directions in each block are statistically analyzed to form a 128-dimensional feature vector.

3. The fault location method for detecting floor defects using stitched images as described in claim 2, characterized in that, The specific methods for matching and image stitching are as follows: a. For the first original image in the original image sequence or the first mask image in the mask sequence, use a KD-tree data structure to store the feature description vector in an orderly manner, and then use a fast approximate k-nearest neighbor algorithm to find the nearest neighbor and the second nearest neighbor. b. Calculate the parameters of the transmission transformation matrix based on the feature points; transform the next original image or the mask image using the transmission transformation matrix, and use a polynomial interpolation algorithm to estimate the non-integer coordinate pixel values ​​to obtain the calibrated image; c. Stitch the previous original image with the transformed next original image, or stitch the previous mask image with the transformed next mask image. The overlapping part can be weighted averaged to obtain the stitched image, and then continue to stitch it with the next original image or mask image.

4. A fault location system for detecting floor defects using stitched images according to the method of claim 1, characterized in that, The system includes a window segmentation module (1), an image stitching module (2), a window positioning module (3), and a defect positioning module (4) connected in sequence. The window segmentation module (1) is configured as follows: A series of original images covering the entire floor area are acquired as the original image sequence. The windows in each of the original images are segmented using a trained convolutional neural network to obtain a binary mask image with the same size as the original image. All the mask images are then sorted to obtain a mask sequence. The image stitching module (2) is configured as follows: The adjacent original images are sequentially subjected to feature extraction, matching, and image stitching to obtain a global image; the same method is used to extract features, match, and stitch the mask image to obtain a global mask; the location of the defect is recalculated and located based on the transmission transformation. The window positioning module (3) is configured as follows: Connectivity detection is performed on the global mask to locate all windows on the floor and determine the door number corresponding to each window; The defect location module (4) is configured as follows: The location of the defect after repositioning is found from the global mask, and the window closest to the repositioned location of the defect is found. Based on this, the location of the defect in the overall floor plan is calculated. The location in the overall floor plan includes the unit and floor number, thereby completing the positioning. The specific method for recalculating and locating the defect is as follows: Let the original center position of the defect be (xbreak, ybreak). According to the transmission transformation... The coordinates of the defect were calculated after image stitching. as follows: 。 5. The system as described in claim 4, characterized in that, The image stitching module (2) includes a feature extraction unit (21), a matching stitching unit (22), and a defect relocation unit (23). The feature extraction unit (21) is configured as follows: Gaussian blur with different parameters is used to represent different scale spaces to detect feature points that exist at different scales, thereby detecting extrema. Unstable extrema, including low-contrast extrema and unstable edge response points, are removed. The gradient argument and magnitude of each pixel in the neighborhood of the feature point are calculated, and a histogram is used to statistically analyze the gradient argument to determine the direction of the feature point. The coordinate axes are rotated to reflect the direction of the feature point, centered on the feature point. The gradient magnitude and direction of the pixels in the window are used to divide the pixels in the window into 16 blocks. The histograms of the eight directions in each block are used to form a total of 128-dimensional feature vectors. The matching splicing unit (22) is configured as follows: For the first original image in the original image sequence or the first mask image in the mask sequence, a KD-tree data structure is used to store the feature description vector in an orderly manner, and then the fast approximate k-nearest neighbor algorithm is used to find the nearest and second nearest neighbors. The parameters of the transmission transformation matrix are calculated based on the feature points. The transmission transformation matrix is ​​used to transform the next original image or the mask image, and a polynomial interpolation algorithm is used to estimate the non-integer coordinate pixel values ​​to obtain the calibrated image. The previous original image is stitched together with the transformed next original image, or the previous mask image is stitched together with the transformed next mask image. The overlapping part is weighted and averaged to obtain the stitched image, and then stitched together with the next original image or mask image. The defect relocation unit (23) is configured as follows: Let the original center position of the defect be (xbreak, ybreak). According to the transmission transformation... The coordinates of the defect were calculated after image stitching. as follows: 。

Images