A Method and System for Detecting Quality Defects in Glass Curtain Walls Based on 3D Modeling

Abstract

This invention discloses a method and system for detecting quality defects in glass curtain walls based on 3D modeling, specifically relating to the field of glass curtain wall inspection. The method includes performing multi-view scanning and simultaneous imaging of the glass curtain wall, writing the sampled content formed at the same acquisition time into corresponding frame records, arranging the frame records in ascending order of acquisition time to obtain an original frame sequence, performing coordinate unification and adjacent frame stitching on the sampled content in the original frame sequence, extracting the separating lines and their intersections from the stitching results, and numbering the separating lines according to their projection positions. This invention solves the problem in existing technologies where the same anomaly is difficult to stably correspond to a specific panel or boundary position by first stably mapping the multi-view acquisition results to the actual panel and boundary positions, and then constructing reference surfaces and reference edges based on the mapping results and performing deviation identification.

Description

Technical Field

[0001] This invention relates to the field of glass curtain wall inspection technology, and more specifically, to a method and system for detecting quality defects in glass curtain walls based on three-dimensional modeling. Background Technology

[0002] In the quality inspection of glass curtain walls, existing technologies mainly focus on the identification of defects such as cracks, damage, panel misalignment and structural deformation. Typically, point cloud data and texture images of the curtain wall surface are acquired through laser scanning equipment, binocular cameras, and drones equipped with scanning devices. Then, image processors are used to denoise, register, fuse and 3D model the acquired data, and the defect judgment is completed by combining geometric deviation analysis and texture feature analysis based on the 3D model.

[0003] However, during the final acceptance and maintenance re-inspection of the facade of high-rise buildings, the acquisition equipment is limited by the difficulty of maintaining stable close-range operation for a long time. The acquisition angle will continuously change with the operation path. At the same time, there is a large range of reflection on the glass surface, and the curtain wall panels are often arranged in a repetitive manner. As a result, the same anomaly can easily be associated with adjacent panels, adjacent boundaries, or adjacent joints in different batches of acquisition results. During on-site verification, there are often inconsistencies in the landing points of the two consecutive test results, fluctuations in the dimensional quantification values, and difficulty in accurately pinpointing the rectification location to a specific panel or boundary segment. The root cause is that although the existing processing flow can complete the anomaly identification, it has not first established a stable correspondence between the acquisition results and the actual structural location.

[0004] Therefore, how to stably map the collected results to the actual panel and boundary positions during the multi-view 3D inspection of glass curtain walls before performing defect identification and quantification has become a pressing technical problem that needs to be solved. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a method and system for detecting quality defects in glass curtain walls based on three-dimensional modeling. This method first stably maps multi-view acquisition results to the actual panel and boundary positions, then constructs reference surfaces and reference edges based on the corresponding results and performs deviation recognition, thereby solving the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for detecting quality defects in glass curtain walls based on three-dimensional modeling, comprising:

[0007] S1. Perform multi-view scanning and synchronous shooting on the glass curtain wall, and write the sampled content formed at the same acquisition time into the corresponding frame record. Arrange the frame records in ascending order according to the acquisition time to obtain the original frame sequence.

[0008] S2. Perform coordinate unification and adjacent frame splicing on the sampled content in the original frame sequence. Extract the dividing lines and their intersections from the splicing results. Number the dividing lines according to the order of projection position. Generate a block record by the closed area enclosed by two adjacent vertical dividing lines and two adjacent horizontal dividing lines.

[0009] S3. For each plate record, read the sampling points inside the plate, calculate the perpendicular distance from each sampling point to the four side boundaries, and write the local coordinates according to the ratio of the left and right perpendicular distances and the ratio of the top and bottom perpendicular distances. Then, merge the sampling points with the same plate identifier and the closest local coordinates in different frame records into the same position record to obtain the position correspondence table.

[0010] S4. Using the position correspondence table as an index, perform position mean and texture synthesis on the same position record to generate reference surface and reference edge. Write the normal difference between the sampling point and the reference surface and the boundary height difference between the corresponding sampling points on both sides of the boundary into the deviation record to obtain the deviation record set.

[0011] S5. The deviation record set is continuously merged according to the plate identifier, boundary identifier and local coordinate order to form the plate deviation segment and the boundary deviation segment. Crack defects and deformation defects are identified according to the local coordinate change direction and normal difference distribution of the plate deviation segment. Misalignment defects are identified according to the boundary height difference distribution of the boundary deviation segment. Defect records are output.

[0012] In a preferred embodiment, S1 includes:

[0013] S1-1. Convert each scanning result and each shooting result to the same time axis. For each scanning result, read the shooting result with the first absolute value of the time difference. Then, read the scanning result with the first absolute value of the time difference for the shooting result. Write the scanning result and the shooting result corresponding to the same scanning result from the two reading results into the same candidate frame record to obtain the candidate frame record set.

[0014] S1-2. For each candidate frame record in the candidate frame record set, the scanning result is projected onto the shooting result using the acquisition position and acquisition posture. The corresponding pixel content is read according to the projection position of each scanning point, and the coordinates of each scanning point and the corresponding pixel content are written into the candidate frame record in the order of the scanning points to obtain the frame record set.

[0015] S1-3. Arrange the frame record set in ascending order of acquisition time, and arrange the frame records with the same acquisition time in the order of rotation along the outer contour of the glass curtain wall to obtain the original frame sequence.

[0016] In a preferred embodiment, S2 includes:

[0017] S2-1. Read each frame record in the original frame sequence. Using the sampling content of the first frame record as a reference, convert the sampling content of the remaining frame records to the coordinate system of the first frame record according to the acquisition position and acquisition posture to obtain a unified sampling sequence.

[0018] S2-2. Perform an overlap region search on the sampling content of two adjacent frames in the unified sampling sequence, take the average coordinates of the sampling points corresponding to the positions in the overlap region and write them into the previous frame record, and merge the sampling points in the non-overlapping region into the previous frame record in sequence to obtain the spliced ​​sequence.

[0019] In a preferred embodiment, S2 further includes:

[0020] S2-3. Expand the sampled content in the splicing sequence according to the same projection direction, connect the consecutive edge sampling points of the projection position to form candidate polylines, merge the candidate polylines that are connected end to end and located on the same straight line into dividing lines, and solve the intersection points between each dividing line.

[0021] S2-4. Divide the dividing lines into vertical dividing lines and horizontal dividing lines according to their projection positions, number them sequentially, and generate a block record by forming a closed area enclosed by two adjacent vertical dividing lines and two adjacent horizontal dividing lines.

[0022] In a preferred embodiment, S3 includes:

[0023] S3-1. For each block record, read the sampling points within the block, calculate the perpendicular distance from each sampling point to the left boundary, right boundary, upper boundary, and lower boundary, write the ratio of the perpendicular distance from the left boundary to the sum of the left and right perpendicular distances into the local horizontal coordinate, write the ratio of the perpendicular distance from the upper boundary to the sum of the upper and lower perpendicular distances into the local vertical coordinate, and write the boundary names corresponding to the perpendicular distances of the four sides in ascending order into the boundary order to obtain the local point record set.

[0024] S3-2. For each local point record in the local point record set, read the other local point records with the same plate identifier, calculate the local horizontal coordinate difference, local vertical coordinate difference, and boundary order comparison results respectively, and write the local point records with the same boundary order and whose sum of local horizontal coordinate difference and local vertical coordinate difference is first into the preferred record. Then write the two local point records that are preferred records to each other into the candidate position record. Otherwise, write the local point records that are not preferred records to each other into the position record to be inspected, and obtain the candidate position record set and the position record set to be inspected.

[0025] S3-3. For each candidate position record in the candidate position record set, read the local point records with the same plate identifier forward and backward according to the frame record order. Merge the local point records that are preferred records of the current candidate position record and have the same boundary order into the same position chain. Write the local point records that only meet the boundary order into the position record to be inspected. Write the local point records with different boundary orders into the disconnected record, and obtain the position chain set, the position record set to be inspected, and the disconnected record set.

[0026] In a preferred embodiment, S3 further includes:

[0027] S3-4. For each location record to be inspected in the location record set, read the location chain records adjacent to it under the same block identifier, calculate the local abscissa difference and local ordinate difference between the location record to be inspected and the previous location chain record, and the local abscissa difference and local ordinate difference between the location record to be inspected and the next location chain record. Merge the location records to be inspected with the same difference direction and the same boundary order into the corresponding location chain. Otherwise, rewrite the location record to be inspected as a disconnected record to obtain the corrected location chain set.

[0028] S3-5. For each position chain in the correction position chain set, retain the position chains with continuous frame record order and consistent local horizontal coordinate change direction and consistent local vertical coordinate change direction. Delete the position chains other than the first one in ascending order of the sum of the cumulative local horizontal coordinate difference and the cumulative local vertical coordinate difference, and then merge the retained position chains into the same position record to obtain the position correspondence table.

[0029] In a preferred embodiment, S4 includes:

[0030] S4-1. Read each record of the same position in the position correspondence table, calculate the position mean and texture mean in each record of the same position, and write the position mean and texture mean into the reference point record to obtain the reference point record set;

[0031] S4-2. Group the reference point record set by plate identifier, connect the reference point records with the same local x-coordinate under the same plate identifier in sequence by local y-coordinate, connect the reference point records with the same local y-coordinate in sequence by local x-coordinate, and form a reference surface by the adjacent connection results, and form a reference edge line by the connection results at the boundary position.

[0032] In a preferred embodiment, S4 further includes:

[0033] S4-3. For each sampling point, read the reference point record with the same plate identifier that surrounds the local coordinates of the sampling point, solve the surface equation of the corresponding reference surface, calculate the signed perpendicular distance from the sampling point to the reference surface, and write the signed perpendicular distance into the normal difference value.

[0034] S4-4. For each reference edge line, read the sampling points with the same local coordinates on both sides of the boundary, calculate the coordinate difference between the sampling points on both sides in the normal direction of the reference edge line, write the coordinate difference into the boundary height difference, and then write the normal difference and the boundary height difference into the deviation record to obtain the deviation record set.

[0035] In a preferred embodiment, S5 includes:

[0036] S5-1. Read each deviation record in the deviation record set, arrange them in the order of plate identifier, boundary identifier, and local coordinates, group the deviation records with empty boundary identifiers, connected local coordinates, and the same sign of normal difference into intra-plate deviation segments, and group the deviation records with the same boundary identifiers, connected local coordinates, and the same sign of boundary height difference into boundary deviation segments, to obtain the deviation segment set.

[0037] S5-2. Connect the plate deviation segments in the deviation segment set according to the local coordinate order, calculate the local horizontal coordinate increment and local vertical coordinate increment of adjacent deviation records, and write the local horizontal coordinate increment sign and local vertical coordinate increment sign of each adjacent deviation record into the direction code. Plate deviation segments with only one value of the direction code are recorded as crack defects, otherwise plate deviation segments with two or more values ​​of the direction code are recorded as deformation defects. At the same time, each boundary deviation segment is recorded as a misalignment defect, thus obtaining the defect segment set.

[0038] S5-3. Write each defect segment in the defect segment set into a defect record according to the plate identifier, boundary identifier, and local coordinate range. Merge defect segments with contiguous local coordinates and the same defect category into the same defect record and output the defect record.

[0039] In a preferred embodiment, a glass curtain wall quality defect detection system based on three-dimensional modeling includes:

[0040] The time-series archiving module is used to perform multi-view scanning and synchronous shooting of the glass curtain wall, and write the sampled content formed at the same acquisition time into the corresponding frame record. The frame records are arranged in ascending order according to the acquisition time to obtain the original frame sequence.

[0041] The segment generation module is used to perform coordinate unification and adjacent frame splicing on the sampled content in the original frame sequence. It extracts the dividing lines and their intersections in the splicing result, numbers the dividing lines according to the order of projection position, and generates segment records by the closed area enclosed by two adjacent vertical dividing lines and two adjacent horizontal dividing lines.

[0042] The position merging module reads the sampling points within the plate for each plate record, calculates the perpendicular distance from each sampling point to the four side boundaries, and writes the local coordinates according to the ratio of the left and right perpendicular distances and the ratio of the top and bottom perpendicular distances. Then, it merges the sampling points with the same plate identifier and the closest local coordinates in different frame records into the same position record to obtain the position correspondence table.

[0043] The benchmark generation module generates a reference surface and reference edge by performing position mean and texture synthesis on the same position record using the position correspondence table as an index. It also writes the normal difference between the sampling point and the reference surface and the boundary height difference between the corresponding sampling points on both sides of the boundary into the deviation record to obtain the deviation record set.

[0044] The defect output module continuously merges the deviation record set according to the plate identifier, boundary identifier, and local coordinate order to form the plate deviation segment and the boundary deviation segment. It identifies crack defects and deformation defects based on the local coordinate change direction and normal difference distribution of the plate deviation segment, and identifies misalignment defects based on the boundary height difference distribution of the boundary deviation segment, and outputs the defect record.

[0045] The technical effects and advantages of this invention are as follows:

[0046] 1. By first stably mapping the multi-view sampling content to the real plate position and the real boundary position, and then carrying out subsequent reference surface construction and deviation identification, it is possible to relatively reduce the situation where the same anomaly falls on adjacent plates, adjacent boundaries or adjacent seams in different batches of inspection, thereby improving the consistency of defect landing points and alleviating the problem of difficulty in determining the rectification location;

[0047] 2. By unifying the original frame sequence to the coordinate system of the first frame record and performing overlapping area retrieval and frame-by-frame stitching on adjacent frames, the sampling content of multiple scattered frames can be transformed into a continuous overall sampling result, thus providing a continuous data foundation for the extraction of dividing lines and the generation of plate records.

[0048] 3. By calculating the perpendicular distance from the sampling point to the four side boundaries in each block record and writing the local horizontal coordinate, local vertical coordinate and boundary order, and then combining the mutual first choice, position chain expansion and correction merging to form a position correspondence table, the erroneous merging of adjacent positions and the splitting of the same position are relatively suppressed.

[0049] 4. By calculating the position average and texture average of the same location records to generate reference point records, and further forming reference surfaces and reference edges, the undulations of the board surface and the misalignment of the boundary can be uniformly converted into normal difference and boundary height difference, thereby improving the comparability of defect quantification results;

[0050] 5. By continuously merging deviation records according to plate identifier, boundary identifier, and local coordinate sequence, and forming intra-plate deviation segments and boundary deviation segments respectively, it is possible to distinguish crack defects, deformation defects, and misalignment defects under the same recording system, thereby relatively improving the structuring degree of defect output results. Attached Figure Description

[0051] Figure 1 This is a flowchart of the method steps of the present invention.

[0052] Figure 2 This is a schematic diagram of the system modules of the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Refer to the instruction manual appendix Figure 1-2 The present invention provides a method for detecting quality defects in glass curtain walls based on three-dimensional modeling, comprising:

[0055] S1. Perform multi-view scanning and synchronous shooting on the glass curtain wall, and write the sampled content formed at the same acquisition time into the corresponding frame record. Arrange the frame records in ascending order according to the acquisition time to obtain the original frame sequence.

[0056] In this specific implementation, S1 addresses the framing issue of the original sampled content. When acquiring data from the outside of the glass curtain wall, the scanning results and the shooting results are usually output by different devices. Although they correspond to the same acquisition process, their output times do not completely overlap, and multiple surround viewpoints often exist at the same time. If the subsequent steps are directly proceeded according to the original output order, problems such as mismatch between scanning and shooting results, confusion in the order of multiple viewpoints at the same moment, and unstable splicing order of adjacent frames are likely to occur. Therefore, each scanning result and each shooting result is first converted to the same time axis, and then bidirectional first-order reading is used to form candidate frame records. Subsequently, each scanning point in the scanning result is projected onto the corresponding shooting result, and the pixel content of each scanning point is added in. Finally, the original frame sequence is organized according to the acquisition time and the surround view order at the same moment.

[0057] The implementation process includes the following steps:

[0058] First, all scan results and imaging results are converted to the same timeline. Specifically, the start time of this detection task is used as a unified starting point. The original acquisition time of each scan result is subtracted from this starting point to obtain the unified time of the scan results. Similarly, the original acquisition time of each imaging result is subtracted from this starting point to obtain the unified time of the imaging results. Then, each scan result is read sequentially according to its original output order. For each scan result, the absolute value of the time difference between it and all imaging results is calculated. The imaging result with the highest absolute value of the time difference is taken as the corresponding imaging result for the current scan result. Then, using this corresponding imaging result as the current object, the time difference between it and all scan results is calculated in reverse. The absolute value of the time difference is used to determine the scanning result that is first in the sequence. If the reverse corresponding scanning result is the same as the current scanning result, then the current scanning result and the corresponding shooting result are written into the same candidate frame record; otherwise, they are not written. If there are two or more shooting results with the same absolute value of time difference, the first one is retained according to the original output order of the shooting results. After this processing, each candidate frame record in the candidate frame record set consists of one scanning result and one shooting result, and there is no situation where one scanning result corresponds to multiple shooting results.

[0059] Next, pixel content from the sampling content is added to each candidate frame record in the candidate frame record set. Specifically, the acquisition position and attitude corresponding to the candidate frame record are read, the coordinates of each scan point in the scan result are converted to the imaging coordinate system corresponding to the shooting result, and then the coordinates of each scan point are projected onto the shooting result to obtain the projected position of each scan point. For scan points whose projected positions fall within the pixel range of the shooting result, the pixel content at that projected position is read, and the scan point coordinates and corresponding pixel content are written into the candidate frame record in scan point order. For scan points whose projected positions fall outside the pixel range of the shooting result, the corresponding pixel content is marked as null and the scan point coordinates are retained. The scan point order here adopts the original output order in the scan result to ensure that the coordinates of each scan point and the pixel content within the same candidate frame record are consistent. If the projected position of a scan point is between two pixels, the corresponding pixel content is read after rounding the horizontal and vertical coordinates respectively. If the rounded position exceeds the pixel range, it is still written as a null value. After processing, the frame record set is obtained.

[0060] Subsequently, the frame record set is organized into an original frame sequence. First, all frame records are sorted in ascending order by acquisition time. For frame records with the same acquisition time, they are then sorted according to the rotation order of the acquisition position along the outer contour of the glass curtain wall. Specifically, each acquisition position with the same acquisition time is first projected onto the plane of the glass curtain wall. The maximum, minimum, maximum, and minimum values ​​of the projected points are read. The average of the maximum and minimum values ​​is used as the center x-coordinate, and the average of the maximum and minimum values ​​is used as the center y-coordinate, thus obtaining the center point of this acquisition position group. Then, using this center point as a reference, the rotation angle value of each acquisition position relative to the center point is calculated and sorted in ascending order by rotation angle value. If two frame records have the same rotation angle value, they are sorted in ascending order by distance from the acquisition position to the center point. If the distances are the same, they are sorted according to the order in which the frame records were written. After sorting, the results are sequentially written into the original frame sequence.

[0061] After the above processing, each original frame record simultaneously fixes three types of relationships: first, the correspondence between the scan result and the shooting result; second, the correspondence between the scan point coordinates and the pixel content; and third, the time order and wrapping order of the frame record in the entire set of acquisition results. In this way, when reading the original frame sequence later, it is not necessary to re-determine which scan result corresponds to which shooting result, nor is it necessary to reorganize the order of multiple views at the same moment. It can directly enter the coordinate unification and adjacent frame stitching. For scenes such as glass curtain walls where the panels are repeated and the viewpoint changes continuously, this step first stabilizes the frame recording order, so that the subsequent extraction of dividing lines and panel generation will not be repeatedly offset due to the instability of the previous frame formation.

[0062] In practical applications: When the detection equipment moves around the building facade, the scanning device continuously outputs the scanning results, and the imaging device outputs the imaging results simultaneously. The system first completes bidirectional first-to-first pairing according to a unified time axis, then projects each scanning point onto the corresponding image to fill in the pixel content, and finally arranges multiple frame records at the same acquisition time in sequence according to the surrounding position, thereby forming an original frame sequence that can directly participate in subsequent stitching.

[0063] S2. Perform coordinate unification and adjacent frame splicing on the sampled content in the original frame sequence. Extract the dividing lines and their intersections from the splicing results. Number the dividing lines according to the order of projection position. Generate a block record by the closed area enclosed by two adjacent vertical dividing lines and two adjacent horizontal dividing lines.

[0064] In this specific implementation, S2 handles the unification, splicing, and segment placement of the original frame sequence. After processing by S1, each frame record has a fixed acquisition time order and surround order, but the sampled content in different frame records is still in its own acquisition position and acquisition posture. If the subsequent position correspondence steps are directly entered, the point range, boundary direction, and separation relationship of the same segment in adjacent frames will still be inconsistent, making it impossible to stably form segment records. Therefore, the first frame record is used as a unified coordinate reference, and the sampled content of the remaining frame records is converted to the coordinate system of the first frame record. Then, the overlapping area is searched and spliced ​​on the unified sampled content of two adjacent frames to form a continuous splicing sequence. Subsequently, edge sampling points are extracted from the splicing sequence and merged to form separating lines. Finally, segment records are generated based on the enclosing relationship between the separating lines.

[0065] The implementation process includes the following steps:

[0066] First, each frame record in the original frame sequence is read. Using the sampling content of the first frame record as a unified reference, the sampling content of the remaining frame records is converted to the coordinate system of the first frame record. Specifically, the first frame record is taken from the first frame record in the original frame sequence; this first position is determined by the acquisition time order and the simultaneous orbiting order. Then, the acquisition position and acquisition posture in the first frame record are read as the reference position and reference orientation of the first frame record coordinate system. Next, the acquisition positions and acquisition postures of the remaining frame records are read sequentially, and the displacement and rotation of the current frame record relative to the first frame record are calculated. The data from the current frame record is then used to... The coordinates of each sampling point are converted to the coordinate system of the first frame record according to the displacement and rotation. During the conversion, the coordinates of the current sampling point are first rotated to a unified orientation according to the current frame record acquisition posture, and then translated to the coordinate system of the first frame record according to the displacement between the current frame record and the first frame record. After the conversion is completed, the sampling content in the first frame record is directly written into the unified sampling sequence, and the converted sampling content in the remaining frame records is written into the unified sampling sequence in the original frame order. After this processing, the coordinates of all sampling points in the unified sampling sequence fall under the same coordinate reference, and subsequent inter-frame stitching can be directly performed according to the positional relationship.

[0067] Next, an overlap region search is performed on the sampling content of two adjacent frames in the unified sampling sequence to obtain a spliced ​​sequence. Specifically, the preceding and following frames are read sequentially according to the order in the unified sampling sequence, and the horizontal and vertical coordinate ranges of the sampling points in both frames are calculated. The intersection of the horizontal and vertical coordinate ranges is defined as the overlap region of the two frames. Then, sampling points falling into this overlap region are read from both frames. For each overlapping region sampling point in the preceding frame, the corresponding sampling point in the following frame is searched in the order of horizontal and vertical coordinates. If a corresponding sampling point is found, the average of the horizontal, vertical, and depth coordinates of both is taken, and the average result is written to the corresponding point in the preceding frame. If no corresponding sampling point is found... For sample points, the original sample points in the previous frame record are retained unchanged. For sample points in the subsequent frame record that are outside the overlapping area, they are sequentially merged into the end of the previous frame record according to the original order in the subsequent frame record. After processing the current two adjacent frames, the updated previous frame record is written into the splicing sequence, and then the updated previous frame record is used as the new previous frame record, and the same processing is performed on the next frame record. By advancing frame by frame, the splicing sequence is finally obtained. The position correspondence here adopts the same point position retrieval under a unified coordinate system, that is, the sample points in the current frame and the subsequent frame that completely correspond to the horizontal coordinate, vertical coordinate and depth coordinate are recorded as position correspondences. If the same sample point in the previous frame corresponds to multiple sample points in the subsequent frames, the first one is taken according to the original writing order of the sample points in the subsequent frames. After this processing, the sampled content in the splicing sequence has been transformed from scattered multiple frames into a continuous overall sampling result.

[0068] Subsequently, the separating lines and their intersections are extracted from the splicing sequence. Specifically, the sampled content in the splicing sequence is first unfolded onto the plane of the curtain wall along the same projection direction. The projection direction is the direction perpendicular to the curtain wall surface in the coordinate system recorded in the first frame. After projection, the planar position of each sampling point is retained. Then, edge sampling points are read from the projection result. Here, edge sampling points refer to sampling points located at the outer edge of the sampling point distribution after projection unfolding. The specific method is as follows: for each sampling point, its horizontal and vertical adjacent points are read. If there are no adjacent points on one horizontal or vertical side, the sampling point is recorded as an edge sampling point. After obtaining the edge sampling points, consecutive edge sampling points at the projection positions are connected sequentially to form candidate polylines. Here, the projection position... "Continuity" means that the subsequent edge sampling point is adjacent to the previous edge sampling point in the horizontal or vertical direction. Then, the candidate polylines are merged: if the endpoints and starting points of two candidate polylines coincide, and all points of the two candidate polylines fall in the same straight line direction after connecting all points of the two candidate polylines, then these two candidate polylines are merged into a single dividing line. After this processing, the separation relationship between the segments in the splicing sequence is restored to multiple dividing lines. After the dividing lines are generated, the intersection points between any two intersecting dividing lines are solved one by one. The coordinates of the intersection points are obtained by solving the linear expressions of the two dividing lines. If the two dividing lines are parallel, no intersection point is generated. After this processing, the separation framework in the splicing sequence is clearly written out, providing a boundary basis for the generation of segment records.

[0069] Finally, the separating lines are distinguished by direction, numbered, and segmented records are generated. Specifically, the direction vectors of each separating line in the projection plane are read first. Separating lines whose absolute value of the vertical component is greater than that of the horizontal component are recorded as vertical separating lines, and those whose absolute value of the horizontal component is greater than that of the vertical component are recorded as horizontal separating lines. If the two are equal, they are recorded as horizontal separating lines. After direction distinction, all vertical separating lines are numbered in ascending order of horizontal projection position, and all horizontal separating lines are numbered in ascending order of vertical projection position. Then, adjacent vertical and horizontal separating lines are read sequentially, and the four intersection points formed between each pair of these four separating lines are determined. These intersection points are then enclosed in the order of upper left, upper right, lower right, and lower left to form a closed region. This closed region is recorded as a segmented record, containing the corresponding segment identifier, four side boundaries, and boundary identifier. If a group of adjacent separating lines does not form a closed quadrilateral region, no segmented record is generated. Thus, the overall sampled content in the splicing sequence is divided into multiple segmented records with clear boundaries. Subsequently, local coordinates can be calculated based on these segmented records, and a position correspondence table can be generated.

[0070] Through the above processing, the discrete sampling content in the original frame sequence is first unified to the coordinate system of the first frame record, and then the adjacent frames are spliced ​​to form a continuous sampling coverage. Then, the dividing lines and their intersections are extracted from the edge sampling points, and finally the entire curtain wall sampling result is divided into the segment records that can directly participate in the subsequent positioning and deviation calculation.

[0071] In practical applications: For a curtain wall facade formed by splicing multiple standard glass panels, after the detection equipment continuously moves along the curtain wall to acquire the original frame sequence, it first converts all frame records to the coordinate system of the first frame record, then takes the average coordinate of the sampling points in the overlapping area of ​​adjacent frames and splices them into the previous frame record, thus forming a continuous splicing sequence; then, it extracts the outer edge sampling points from the splicing result after projection and connects them to form candidate polylines, and merges the candidate polylines that are connected end to end and located on the same straight line into dividing lines; then, it generates panel records based on the closed area enclosed by the vertical dividing lines and the horizontal dividing lines, which can divide the entire curtain wall into multiple panel areas with clear boundaries for subsequent panel sampling point positioning and defect detection.

[0072] S3. For each plate record, read the sampling points inside the plate, calculate the perpendicular distance from each sampling point to the four side boundaries, and write the local coordinates according to the ratio of the left and right perpendicular distances and the ratio of the top and bottom perpendicular distances. Then, merge the sampling points with the same plate identifier and the closest local coordinates in different frame records into the same position record to obtain the position correspondence table.

[0073] In this specific implementation, S3 addresses the stable correspondence between sampling points within a plate and records in different frames. After processing by S2, each plate record has clearly defined four-sided boundaries and the range of sampling points within the plate. However, sampling points of the same plate in different frame records still originate from different acquisition positions and different acquisition postures. If they are directly merged according to spatial distance, it is easy to mistakenly merge sampling points at adjacent positions into the same position, or split sampling points at the same position in different frames into multiple correspondences. Therefore, the sampling points within the plate are first converted into local abscissas and local ordinates relative to the four-sided boundaries within each plate record, and the boundary order is formed by the order of the vertical distances of the four-sided boundaries. Then, the inter-frame correspondence is established by using the plate identifier, local coordinates, and boundary order. After the initial correspondence is formed, the position chain continues to be extended forward and backward according to the frame record order, and forward and backward corrections are performed on position records that are not directly merged. Finally, multiple position chains that share the same local point record are adjudicated to obtain a position correspondence table.

[0074] The implementation process includes the following steps:

[0075] For each segment record, first read the sampling points within that segment record, and then calculate the perpendicular distance to the left, right, top, and bottom boundaries for each point. The left, right, top, and bottom boundaries are all taken from the four side boundaries of the corresponding segment record, and the perpendicular distance is the length of the perpendicular line from the sampling point to the corresponding boundary. If a sampling point falls on one of the four side boundaries, it is not considered a segment sampling point for the current position but is retained according to its boundary identifier for subsequent reference edge line and boundary height difference calculations. After obtaining the perpendicular distance, the result of dividing the left boundary perpendicular distance by the sum of the left and right boundary perpendicular distances is written into the local abscissa, and the result of dividing the top boundary perpendicular distance by... The sum of the vertical distances of the upper and lower boundaries is written into the local ordinate. Then, the vertical distances of the four sides are arranged in ascending order of value, and the corresponding boundary names are written into the boundary order in sequence. If two vertical distance values ​​are the same, they are written in the fixed order of left boundary, right boundary, upper boundary, and lower boundary. After the above processing is completed, the plate identifier, frame record identifier, local abscissa, local ordinate, boundary order, and original position of the sampling point are written into the same local point record to form a local point record set. Through this processing, the sampling points in the plate in different frame records are compressed into the relative position expression of the same plate, and the subsequent correspondence no longer directly depends on the original spatial coordinates.

[0076] After the local point record set is formed, a preferred record is established for each local point record. Specifically, the remaining local point records with the same module identifier are read, and the local x-coordinate difference and local y-coordinate difference between the current local point record and each of the other local point records are calculated. Then, the boundary order is compared item by item to see if it is consistent. Only local point records with consistent boundary order continue to participate in the preferred record determination. For local point records that satisfy the condition of consistent boundary order, the sum of the local x-coordinate difference and the local y-coordinate difference is calculated first, and they are sorted in ascending order of this sum. If the sums are the same, the local x-coordinate differences are compared first, with the smaller local x-coordinate difference appearing first. If the local x-coordinate differences are still the same, the local y-coordinate differences are compared, with the smaller local y-coordinate difference appearing first. If the local ordinate differences are still the same, the first one is selected according to the frame recording order. The first local point record is written as the preferred record of the current local point record. Subsequently, the preferred record relationship is read one by one. If two local point records are each other's preferred records, these two local point records are written into the same candidate position record. If the current local point record has a preferred record, but there is no mutual preferred relationship between it and the preferred record, the current local point record is written into the position record to be inspected. After this processing, a candidate position record set and a position record set to be inspected are formed. The candidate position records obtained in this way are not simple nearest neighbor results, but bidirectional correspondence results that simultaneously satisfy the consistency of plate identifier, the consistency of boundary order, and mutual preferredness. They can be used as the starting point for subsequent position chain expansion.

[0077] After the candidate location record set is generated, a location chain is constructed for each candidate location record. During execution, starting from the frame record containing the current candidate location record, local point records with the same plate identifier are read forward frame by frame in frame record order, and then local point records with the same plate identifier are read backward frame by frame in frame record order. Whether reading forward or backward, the local point record at the end of the current location chain is taken as the current record. Local point records in the adjacent previous or next frame are read, and the preferred relationship with the current record is determined according to the same preferred selection rule in S3-2. If the read local point record and the current record are preferred records for each other and have the same boundary order, the local point record is merged into the same location chain. If the local point records obtained only satisfy the condition that the boundary order is the same, but do not form a mutually preferred relationship, then the local point record is written into the position record to be inspected; if the boundary order of the read local point records is different, then the local point record is written into the disconnected record, and reading along that direction is stopped; if no local point records with the same plate identifier are read in adjacent frames in a certain direction, reading in that direction is also stopped; by expanding frame by frame forward and frame by frame backward, each candidate position record is organized into a position chain, and a position chain set, a position record set to be inspected set, and a disconnected record set are formed simultaneously; the disconnected records here no longer participate in the continued expansion of the current corresponding position chain, but retain their record content for subsequent independent position merging or use as unmatched points;

[0078] After the position chain is formed, the position records to be inspected are corrected before and after. Specifically, for each position record to be inspected, the position chain records adjacent to it under the same block identifier are read first. The preceding position chain record is the position chain record that is closest to the position record to be inspected in the frame recording order and located before it. The following position chain record is the position chain record that is closest to the position record to be inspected in the frame recording order and located after it. Then, the local abscissa difference and local ordinate difference between the position record to be inspected and the preceding position chain record, as well as the local abscissa difference and local ordinate difference between the position record to be inspected and the following position chain record, are calculated respectively, and the direction of change of these two sets of differences is read respectively. The direction of change is determined by the result of subtracting the preceding value from the following value. A value greater than zero is recorded as positive, a value less than zero is recorded as negative, and a value equal to zero is recorded as zero. If the position record to be inspected is... If the local horizontal and vertical coordinate difference directions between the detected position record and the previous position chain record are consistent with the local horizontal and vertical coordinate difference directions between the detected position record and the next position chain record, and the boundary order of the detected position record is the same as that of the position chain records on both sides, then the detected position record is merged into the corresponding position chain; otherwise, the detected position record is rewritten as a disconnected record. If the detected position record exists only in the previous position chain record or only in the next position chain record, then merging is not performed, and it is directly rewritten as a disconnected record. After this processing, a corrected position chain set is obtained. The purpose of this setting is to add those position records that are not mutually preferred but maintain consistency in the propagation direction before and after the frame record into the correct position chain, while removing position records with interrupted direction changes or altered boundary order.

[0079] Finally, retention and adjudication are performed on each position chain in the corrected position chain set to generate a position correspondence table. Specifically, each position chain's local point records are read sequentially to check if their frame records are written continuously. Continuity here means that the frame record corresponding to the next local point record in the position chain is located after the previous local point record, and there are no other frame records missing in between. Then, the local abscissa difference and local ordinate difference between adjacent local point records in the position chain are calculated sequentially, and the direction of change of each difference is read. If the direction of change of the local abscissa and the direction of change of the local ordinate are consistent throughout the entire position chain, the position chain is retained; otherwise, it is deleted. Next, each retained position chain is checked to see if it shares the same local point record. If two or more position chains share the same local point record... The process involves calculating the sum of the cumulative difference between the local x-coordinate and the cumulative difference between the local y-coordinate in each position chain, sorting them in ascending order of this sum, retaining only the position chain at the top, and deleting all others. If the sum is the same during sorting, the cumulative difference between the local x-coordinates is compared first, and the smaller one is retained. If they are still the same, the cumulative difference between the local y-coordinates is compared, and the smaller one is retained. If they are still the same, the position chain is sorted by the frame record order of the first local point record. After deletion, each retained position chain is merged into a single position record, and the plate identifier, the frame record range included in the position chain, the local x-coordinate, the local y-coordinate, and the original position of the corresponding sampling point are written into the position correspondence table. At this point, the stable correspondence of the same plate in different frame records is fixed, providing a direct index for the subsequent generation of position averages, texture averages, reference surfaces, and reference edges.

[0080] Through the above processing, the sampling points within the plate are no longer directly merged according to their original spatial positions. Instead, they are first compressed into the local abscissa, local ordinate, and boundary order expression within the same plate. Then, through preferred correspondence, position chain expansion, pre- and post-correction, and co-point adjudication, the sampling point groups with stable cross-frame correspondences are gradually screened out, and finally a position correspondence table is formed. This avoids the erroneous merging of adjacent positions and also avoids the same real position being split into multiple sets of correspondences, so that the subsequent reference surface construction is based on the same position record with stable position and inter-frame closure.

[0081] In practical applications: For a standard rectangular glass plate, the original coordinates of its sampling points will differ across multiple frame records due to changes in the sampling angle. The system first calculates the perpendicular distances from these sampling points to the four side boundaries and writes the local abscissa, local ordinate, and boundary order. Then, under the same plate identifier, it first filters out local point records with consistent boundary order and mutual preference as candidate position records, and then expands the position chain forward and backward according to the frame record order. For position records that are not directly merged, it reads the adjacent position chains before and after them and checks whether the directions of the local abscissa difference and the local ordinate difference are consistent. If they are consistent, they are merged into the position chain; if they are inconsistent, they are rewritten as disconnected records. Finally, it deletes the non-first position chains that share the same local point record and merges the remaining position chains into the same position record. The resulting position correspondence table can stably lock the corresponding points of the same glass plate in different viewpoints and different frames to the same real position, providing consistent input for subsequent reference surface and deviation record calculations.

[0082] S4. Using the position correspondence table as an index, perform position mean and texture synthesis on the same position record to generate reference surface and reference edge. Write the normal difference between the sampling point and the reference surface and the boundary height difference between the corresponding sampling points on both sides of the boundary into the deviation record to obtain the deviation record set.

[0083] In this specific implementation, S4 deals with the problem of generating reference geometry from the position correspondence table and forming deviation records. After processing in S3, the corresponding sampling points of the same plate in different frame records have been merged into the same position record. However, these same position records are still just discrete points and cannot be directly used to judge plate surface undulations, boundary misalignments, and local deviations. Therefore, the position mean and texture mean of the same position record are first calculated to form a reference point record. Then, the local horizontal coordinate and local vertical coordinate of the reference point record inside the plate are connected to generate a reference surface, and a reference edge line is formed at the boundary position. Subsequently, the normal difference of each sampling point is calculated based on the reference surface, and the boundary height difference of the sampling points on both sides of the boundary is calculated based on the reference edge line. Finally, the normal difference and boundary height difference are uniformly written into the deviation record for subsequent identification of crack defects, deformation defects, and misalignment defects.

[0084] The implementation process includes the following steps:

[0085] First, read each record of the same position from the position mapping table, calculate the position mean and texture mean for each record, and write them to the reference point record. During execution, for each record of the same position, read the original positions of all sample points contained within it, sum the horizontal, vertical, and depth coordinates of each original sample point position, and then divide each sum by the number of sample points in that record to obtain the position mean for that record. Simultaneously, read all pixel content corresponding to that record, sum them according to the same color channel, and then divide each sum by the number of sample points to obtain the texture mean for that record. If the pixel content corresponding to a certain sampling point is marked as null, then the sampling point will not participate in the calculation of the texture mean, but will still participate in the calculation of the position mean. If the pixel content in the entire record of the same position is marked as null, then the texture mean will be written as null. After the position mean and texture mean are obtained, the plate identifier, local x-coordinate, local y-coordinate, position mean and texture mean are written into the same reference point record, forming a reference point record set in sequence. After this processing, the corresponding sampling points that were originally scattered across frames are compressed into reference point records with stable local positions within the plate and unified spatial positions and texture values.

[0086] Next, the reference point record set is grouped according to the plate identifier, and reference surfaces and reference edges are generated under the same plate identifier. Specifically, within each plate group, reference point records with the same local x-coordinate are first read and connected sequentially in ascending order of local y-coordinate to form a vertical connection line; then, reference point records with the same local y-coordinate are read and connected sequentially in ascending order of local x-coordinate to form a horizontal connection line; subsequently, four reference point records with adjacent local x-coordinates and adjacent local y-coordinates are read sequentially, forming a reference surface unit in the order of upper left, upper right, lower right, and lower left, and all reference surface units are written as the reference surface of that plate according to the plate identifier. Here, adjacent local x-coordinates and... Adjacent local ordinates are determined based on the sorting results of reference point records within the same plate group; adjacent records are recorded as adjacent. For reference point records located at the four boundaries of the plate, their adjacent connection results are read, and the connection results at the outer edge of the plate are written as reference edges. If there are multiple consecutive reference point records at a certain boundary position, these reference point records are connected sequentially to form the reference edge of that side. If a reference surface unit is missing the four required reference point records, that reference surface unit is not generated. After this processing, each plate forms a reference surface composed of multiple reference surface units, as well as reference edges corresponding to the four boundaries, providing a geometric reference for subsequent deviation calculations.

[0087] Subsequently, the normal difference is calculated for each sampling point. Specifically, the plate identifier of the sampling point is first read, and then reference point records with the same plate identifier that can enclose the local coordinates of the sampling point are read from the reference point record set. Here, "enclosing" means that there are four reference point records, whose local x-coordinates are located on both sides of the local x-coordinate of the sampling point, and whose local y-coordinates are located on both sides of the local y-coordinate of the sampling point, and the local coordinates of the sampling point fall within the local coordinate range enclosed by these four reference point records. If there are four completely enclosing reference point records, the reference surface unit corresponding to these four reference point records is used as the reference surface of the current sampling point; if there are no four completely enclosing reference point records, the reference surface unit corresponding to these four reference point records is read from the reference point record set. The three nearest reference points to the local coordinates of the sampling point are recorded, and the local reference surface is solved using these three reference points. After the reference surface is determined, the surface equation is first solved using the three non-collinear reference points in the reference surface. Then, the original position of the current sampling point is substituted into the surface equation to obtain the signed vertical distance from the sampling point to the reference surface. The sign of the signed vertical distance is determined by whether the sampling point is on the same side or opposite side of the reference surface normal. It is written as a positive value when it is on the same side of the reference surface normal, a negative value when it is on the opposite side, and a zero value when it is on the reference surface. After obtaining the signed vertical distance, it is written into the normal difference value of the sampling point. In this way, the deviation of each sampling point from the plate reference surface is uniformly expressed as the normal difference value, which can be directly used for merging deviation segments within the plate later.

[0088] Further, the boundary elevation difference is calculated for each reference edge and recorded in the deviation record. Specifically, each reference edge is read individually, and then sampling points on both sides of that reference edge with the same local coordinates are read. Here, "same local coordinates" means that the local x-coordinates and y-coordinates of the sampling points on both sides of the boundary correspond to the same set of local positions, differing only in the two sides of the boundary. Then, the reference edge normal is determined: the direction vector of the reference edge is taken, and the normal vector of the reference surface of the plate containing the reference edge is read. The cross product of this normal vector and the reference edge direction vector yields the reference edge normal. After determining the reference edge normal, the original positions of the sampling points on both sides of the boundary are projected onto the reference edge normal, and the coordinate values ​​of the sampling points on both sides on this normal are obtained. The result of subtracting the coordinates of the other side from the coordinates of one side is written into the boundary elevation difference. If there is only one sampling point at a certain boundary location, the boundary elevation difference is not calculated for that location. After the normal difference and boundary elevation difference are calculated, the corresponding plate identifier, boundary identifier, local abscissa, local ordinate, normal difference, and boundary elevation difference are written for each sampling point to form an deviation record. For sampling points within the plate, the boundary identifier is written as a null value marker. For sampling points on both sides of the boundary, the boundary identifier is written as the boundary identifier of the corresponding reference edge line. After processing all sampling points in sequence, the deviation record set is obtained. At this point, the deviation within the plate and the boundary misalignment are uniformly written into the same record structure, and subsequent deviation segment merging and defect identification can be carried out directly according to the plate identifier, boundary identifier, and local coordinate order.

[0089] Through the above processing, the same location record in the location correspondence table is first compressed into a reference point record, and then the reference point record generates the plate reference surface and reference edge line. Subsequently, the normal deviation of each sampling point relative to the reference surface and the boundary misalignment relative to the reference edge line are uniformly written into the deviation record set. After this processing, the undulation changes within the plate and the elevation changes at the boundary are both converted into quantitative results under the same plate coordinate system. When merging the deviation segments within the plate and the boundary deviation segments in the future, it is no longer necessary to go back and read the original frame records and original points. The defect classification can be completed directly according to the deviation record set.

[0090] In practical applications: For a glass plate with a pre-generated positional mapping table, the system first calculates the positional and texture averages of multiple corresponding points in the same position record to form a reference point record covering the interior of the glass plate. Then, it connects these reference point records according to the local horizontal and vertical coordinates to generate a reference surface and four side reference edges. Next, it reads the reference point record surrounding the local coordinates of each sampling point within the plate, solves the surface equation of the corresponding reference surface, calculates the signed vertical distance from the sampling point to the reference surface, and writes it as the normal difference. At the same time, for the sampling points on both sides of the boundary position, it calculates the coordinate difference along the normal of the reference edge and writes it as the boundary height difference. Finally, it writes the normal difference and boundary height difference into the deviation record set. Through this process, local bulges, local depressions, and boundary misalignments between adjacent plates can all be stably expressed in the same data structure, providing a direct basis for subsequent defect identification.

[0091] S5. The deviation record set is continuously merged according to the plate identifier, boundary identifier and local coordinate order to form the plate deviation segment and the boundary deviation segment. Crack defects and deformation defects are identified according to the local coordinate change direction and normal difference distribution of the plate deviation segment. Misalignment defects are identified according to the boundary height difference distribution of the boundary deviation segment. Defect records are output.

[0092] In this specific implementation, S5 handles the merging, classification, and output of the deviation record set. After processing by S4, the normal difference of each sampling point relative to the reference surface and the boundary height difference relative to the reference edge have been uniformly written into the deviation record set. However, these deviation records are still discretely distributed point results and cannot directly correspond to crack defects, deformation defects, and misalignment defects. Therefore, all deviation records are first organized according to plate identifier, boundary identifier, and local coordinate order, and then in-plate deviation segments and boundary deviation segments are formed respectively. Subsequently, the local horizontal coordinate increment, local vertical coordinate increment, and direction code are calculated for the in-plate deviation segments to distinguish between crack defects and deformation defects, and the boundary deviation segments are directly written as misalignment defects. Finally, contiguous defect segments with the same category are merged into the same defect record to form the final output result.

[0093] The implementation process includes the following steps:

[0094] First, read all deviation records in the deviation record set and arrange them sequentially by plate identifier, boundary identifier, local x-coordinate, and local y-coordinate. In practice, first group by plate identifier in ascending order, then arrange by boundary identifier within each plate group, with deviation records having empty boundary identifiers listed first, and those with non-empty boundary identifiers arranged in ascending order by their original boundary identifier numbers. Under the same boundary identifier, arrange by local x-coordinate in ascending order; if local x-coordinates are the same, arrange by local y-coordinate in ascending order. After sorting, read each deviation record one by one. For deviation records with empty boundary identifiers, check if the next deviation record simultaneously satisfies the following conditions: empty boundary identifier, contiguous local coordinates, and same sign for normal difference. Contiguous local coordinates mean that the local x-coordinates of two deviation records are the same and their local y-coordinates are adjacent in the sorting order, or that their local y-coordinates are the same and their local x-coordinates are adjacent in the sorting order. Same sign for normal difference means that the normal differences of two deviation records are both positive or both are zero. Negative deviation records with a normal difference of zero are not included in the same-sign merging; deviation records that meet the above conditions are successively merged into the same deviation segment within the same board until the conditions are no longer met; for deviation records with the same boundary identifier, check whether the subsequent deviation record simultaneously meets the conditions of the same boundary identifier, contiguous local coordinates, and same-sign boundary height difference; where the same-sign boundary height difference means that the boundary height differences of the two deviation records are both positive or both negative, and deviation records with a boundary height difference of zero are not included in the same-sign merging; deviation records that meet the conditions are successively merged into the same boundary deviation segment until the conditions are no longer met; if the boundary identifier changes, or the local coordinates no longer contiguous, or the normal difference or boundary height difference sign changes, the current deviation segment ends, and the next deviation segment is generated again from the current deviation record; after all readings are completed, the deviation segment set is obtained; after this processing, continuous deviations within the board and continuous misalignments at the boundary form independent segment structures, and are no longer mixed during subsequent classification;

[0095] Next, defect classification is performed on each intra-plate deviation segment and each boundary deviation segment in the deviation segment set. For each intra-plate deviation segment, the deviation records within the segment are first connected in local coordinate order. Then, adjacent two deviation records are read one by one. The local x-coordinate of the latter deviation record is subtracted from the local x-coordinate of the former deviation record to obtain the local x-coordinate increment. The local y-coordinate of the latter deviation record is subtracted from the local y-coordinate of the former deviation record to obtain the local y-coordinate increment. When the local x-coordinate increment is greater than zero, it is recorded as horizontal positive; when it is equal to zero, it is recorded as horizontal zero. When the local y-coordinate increment is greater than zero, it is recorded as vertical positive; when it is equal to zero, it is recorded as vertical zero. Subsequently, the horizontal and vertical values ​​of each group of adjacent deviation records are combined and written into the direction code, where horizontal positive and vertical zero, horizontal zero and vertical positive, and horizontal positive and vertical positive are written as different direction codes. Since the intra-plate deviation segments are in ascending order of local coordinates... Once the connection is formed, the local horizontal and vertical coordinate increments will not be negative. After the direction code is written, the number of direction code values ​​in the current plate deviation segment is counted. If the direction code has only one value, the plate deviation segment is recorded as a crack defect. If the direction code has more than two values, the plate deviation segment is recorded as a deformation defect. For each boundary deviation segment, the direction code is not calculated again, and the boundary deviation segment is directly recorded as a misalignment defect. After classification, the plate identifier, boundary identifier, local coordinate range, and defect category are written into the corresponding defect segment to obtain the defect segment set. After this processing, crack defects, deformation defects, and misalignment defects have clear classification directions. Crack defects correspond to plate deviation segments extending in a single direction, deformation defects correspond to plate deviation segments expanding in multiple directions, and misalignment defects correspond to boundary deviation segments formed by continuous boundary misalignment.

[0096] Finally, the defect segments in the defect segment set are merged and output. Specifically, each defect segment is read individually and written into a defect record according to the plate identifier, boundary identifier, and local coordinate range. The local coordinate range is determined by the local x-coordinate and local y-coordinate ranges of all deviations from the record within the defect segment, specifically written as the starting local x-coordinate, ending local x-coordinate, starting local y-coordinate, and ending local y-coordinate. After the defect record is written, it is checked whether there are adjacent defect segments with contiguous local coordinates and the same defect category under the same plate identifier. If so, these defect segments are merged into the same defect record. During merging, the local coordinate ranges of the same defect record are re-unified, i.e., the starting local x-coordinate is... The minimum value among all defect segments is used, the maximum value among all defect segments is used for the ending local x-coordinate, the minimum value among all defect segments is used for the starting local y-coordinate, and the maximum value among all defect segments is used for the ending local y-coordinate. The panel identifier remains unchanged, and the boundary identifier is retained only if the boundary identifiers of all defect segments are the same; otherwise, it is marked as a null value. If the panel identifiers are different, or the defect categories are different, or the local coordinates are not connected, merging is not performed, and they are retained as independent defect records. After all defect segments are processed, the defect records are output according to the panel identifier, boundary identifier, and local coordinate range. At this point, the deviation record set is transformed into defect record results that can be directly used for subsequent display, alarms, and rectification.

[0097] Through the above processing, the discrete point deviation results in the deviation record set are first merged into deviation segments, then classified into crack defects, deformation defects, and misalignment defects, and finally uniformly written as defect record output. The whole process is consistent under the plate identifier, boundary identifier, and local coordinate system. This not only distinguishes between continuous deviations within the plate and continuous deviations at the boundary, but also places different types of defects into a unified record structure, which is convenient for subsequent direct reading and display by plate, by boundary, and by position range.

[0098] In practical applications: For a glass plate, if the boundary markers of a group of consecutive deviation records within the plate are empty, the normal difference values ​​are all positive, and the direction codes of adjacent deviation records always have the same value, then this group of deviation records is first merged into a deviation segment within the plate and then recorded as a crack defect; if another group of deviation records within the plate also has empty boundary markers, but the direction codes of adjacent deviation records show two or more values ​​successively, then this group of deviation records is recorded as a deformation defect; if the boundary height differences under the same boundary marker between two adjacent glass plates are consecutively of the same sign, then the corresponding deviation records are merged into a boundary deviation segment and recorded as a misalignment defect; then, defect segments with connected local coordinates and consistent defect categories within the same plate are further merged into the same defect record, thus obtaining the defect output result directly corresponding to a specific plate, specific boundary, and specific local location.

[0099] Furthermore, a glass curtain wall quality defect detection system based on 3D modeling includes:

[0100] The time-series archiving module is used to perform multi-view scanning and synchronous shooting of the glass curtain wall, and write the sampled content formed at the same acquisition time into the corresponding frame record. The frame records are arranged in ascending order according to the acquisition time to obtain the original frame sequence.

[0101] The segment generation module is used to perform coordinate unification and adjacent frame splicing on the sampled content in the original frame sequence. It extracts the dividing lines and their intersections in the splicing result, numbers the dividing lines according to the order of projection position, and generates segment records by the closed area enclosed by two adjacent vertical dividing lines and two adjacent horizontal dividing lines.

[0102] The position merging module reads the sampling points within the plate for each plate record, calculates the perpendicular distance from each sampling point to the four side boundaries, and writes the local coordinates according to the ratio of the left and right perpendicular distances and the ratio of the top and bottom perpendicular distances. Then, it merges the sampling points with the same plate identifier and the closest local coordinates in different frame records into the same position record to obtain the position correspondence table.

[0103] The benchmark generation module generates a reference surface and reference edge by performing position mean and texture synthesis on the same position record using the position correspondence table as an index. It also writes the normal difference between the sampling point and the reference surface and the boundary height difference between the corresponding sampling points on both sides of the boundary into the deviation record to obtain the deviation record set.

[0104] The defect output module continuously merges the deviation record set according to the plate identifier, boundary identifier, and local coordinate order to form the plate deviation segment and the boundary deviation segment. It identifies crack defects and deformation defects based on the local coordinate change direction and normal difference distribution of the plate deviation segment, and identifies misalignment defects based on the boundary height difference distribution of the boundary deviation segment, and outputs the defect record.

[0105] Working Principle: This scheme first uses a scanning device and an imaging device to acquire data from multiple perspectives of the glass curtain wall. Then, an image processor pairs the scanning and imaging results along the same timeline to form a raw frame sequence. Subsequently, the sampled content of each frame is unified to the same coordinate system and stitched frame by frame. From the stitched result, the dividing lines and intersections of the curtain wall are extracted to generate a record for each glass panel. Next, within each panel, the sampling points are converted into local coordinates relative to the four side boundaries, and sampling points belonging to the same real location are merged into the same location record between different frames. Based on this, the position of the record at the same location is further calculated. The average value and texture average value are set to form reference points, which are then used to construct reference surfaces and reference edges. Finally, the normal difference between each sampling point and the reference surface, as well as the boundary height difference on both sides of the boundary, are calculated. The discrete deviation results are merged into deviation segments, and crack defects, deformation defects, and misalignment defects are further identified, and defect records are output. This scheme essentially aligns data from different perspectives first, locks the real position within the same plate, and then determines where the deviation occurred based on a unified reference geometry. Therefore, the final defect location and defect category can be stably mapped to specific plates and specific boundaries.

[0106] For example, during the acceptance or re-inspection of glass curtain walls in high-rise buildings, drones fly around the facade, laser scanning devices continuously collect point information, and imaging devices simultaneously acquire images of the curtain wall surface. The image processor first pairs these two types of results into the same frame, and then stitches the results of multiple frames into a continuous sampling result of the entire curtain wall. Next, the system identifies the dividing lines between each glass panel, determines the boundary of each glass panel, and then merges the sampling points from different perspectives falling on the same glass panel and the same local location together to generate a reference point and reference surface for that location. If a thin, continuous deviation appears on a glass surface, it will be identified as a crack defect; if a certain area shows multi-directional undulations on the panel surface, it will be identified as a deformation defect; if two adjacent glass panels have a continuous height difference at a common boundary, it will be identified as a misalignment defect. In this way, the inspectors do not see scattered point cloud anomalies, but defect results that have been correctly matched with specific panels and specific boundary locations, which facilitates direct review and rectification.

[0107] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting quality defects in glass curtain walls based on three-dimensional modeling, characterized in that, include: S1. Perform multi-view scanning and synchronous shooting on the glass curtain wall, and write the sampled content formed at the same acquisition time into the corresponding frame record. Arrange the frame records in ascending order according to the acquisition time to obtain the original frame sequence. S2. Perform coordinate unification and adjacent frame splicing on the sampled content in the original frame sequence. Extract the dividing lines and their intersections from the splicing results. Number the dividing lines according to the order of projection position. Generate a block record by the closed area enclosed by two adjacent vertical dividing lines and two adjacent horizontal dividing lines. S3. For each plate record, read the sampling points inside the plate, calculate the perpendicular distance from each sampling point to the four side boundaries, and write the local coordinates according to the ratio of the left and right perpendicular distances and the ratio of the top and bottom perpendicular distances. Then, merge the sampling points with the same plate identifier and the closest local coordinates in different frame records into the same position record to obtain the position correspondence table. S4. Using the position correspondence table as an index, perform position mean and texture synthesis on the same position record to generate reference surface and reference edge. Write the normal difference between the sampling point and the reference surface and the boundary height difference between the corresponding sampling points on both sides of the boundary into the deviation record to obtain the deviation record set. S5. The deviation record set is continuously merged according to the plate identifier, boundary identifier and local coordinate order to form the plate deviation segment and the boundary deviation segment. Crack defects and deformation defects are identified according to the local coordinate change direction and normal difference distribution of the plate deviation segment. Misalignment defects are identified according to the boundary height difference distribution of the boundary deviation segment. Defect records are output.

2. The method for detecting quality defects in glass curtain walls based on three-dimensional modeling according to claim 1, characterized in that: S1 includes: S1-1. Convert each scanning result and each shooting result to the same time axis. For each scanning result, read the shooting result with the first absolute value of the time difference. Then, read the scanning result with the first absolute value of the time difference for the shooting result. Write the scanning result and the shooting result corresponding to the same scanning result from the two reading results into the same candidate frame record to obtain the candidate frame record set. S1-2. For each candidate frame record in the candidate frame record set, the scanning result is projected onto the shooting result using the acquisition position and acquisition posture. The corresponding pixel content is read according to the projection position of each scanning point, and the coordinates of each scanning point and the corresponding pixel content are written into the candidate frame record in the order of the scanning points to obtain the frame record set. S1-3. Arrange the frame record set in ascending order of acquisition time, and arrange the frame records with the same acquisition time in the order of rotation along the outer contour of the glass curtain wall to obtain the original frame sequence.

3. The method for detecting quality defects in glass curtain walls based on three-dimensional modeling according to claim 2, characterized in that: S2 includes: S2-1. Read each frame record in the original frame sequence. Using the sampling content of the first frame record as a reference, convert the sampling content of the remaining frame records to the coordinate system of the first frame record according to the acquisition position and acquisition posture to obtain a unified sampling sequence. S2-2. Perform an overlap region search on the sampling content of two adjacent frames in the unified sampling sequence, take the average coordinates of the sampling points corresponding to the positions in the overlap region and write them into the previous frame record, and merge the sampling points in the non-overlapping region into the previous frame record in sequence to obtain the spliced ​​sequence.

4. The method for detecting quality defects in glass curtain walls based on three-dimensional modeling according to claim 3, characterized in that: S2 further includes: S2-3. Expand the sampled content in the splicing sequence according to the same projection direction, connect the consecutive edge sampling points of the projection position to form candidate polylines, merge the candidate polylines that are connected end to end and located on the same straight line into dividing lines, and solve the intersection points between each dividing line. S2-4. Divide the dividing lines into vertical dividing lines and horizontal dividing lines according to their projection positions, number them sequentially, and generate a block record by forming a closed area enclosed by two adjacent vertical dividing lines and two adjacent horizontal dividing lines.

5. The method for detecting quality defects in glass curtain walls based on three-dimensional modeling according to claim 4, characterized in that: S3 includes: S3-1. For each block record, read the sampling points within the block, calculate the perpendicular distance from each sampling point to the left boundary, right boundary, upper boundary, and lower boundary, write the ratio of the perpendicular distance from the left boundary to the sum of the left and right perpendicular distances into the local horizontal coordinate, write the ratio of the perpendicular distance from the upper boundary to the sum of the upper and lower perpendicular distances into the local vertical coordinate, and write the boundary names corresponding to the perpendicular distances of the four sides in ascending order into the boundary order to obtain the local point record set. S3-2. For each local point record in the local point record set, read the other local point records with the same plate identifier, calculate the local horizontal coordinate difference, local vertical coordinate difference, and boundary order comparison results respectively, and write the local point records with the same boundary order and whose sum of local horizontal coordinate difference and local vertical coordinate difference is first into the preferred record. Then write the two local point records that are preferred records to each other into the candidate position record. Otherwise, write the local point records that are not preferred records to each other into the position record to be inspected, and obtain the candidate position record set and the position record set to be inspected. S3-3. For each candidate position record in the candidate position record set, read the local point records with the same plate identifier forward and backward according to the frame record order. Merge the local point records that are preferred records of the current candidate position record and have the same boundary order into the same position chain. Write the local point records that only meet the boundary order into the position record to be inspected. Write the local point records with different boundary orders into the disconnected record, and obtain the position chain set, the position record set to be inspected, and the disconnected record set.

6. The method for detecting quality defects in glass curtain walls based on three-dimensional modeling according to claim 5, characterized in that: S3 further includes: S3-4. For each location record to be inspected in the location record set, read the location chain records adjacent to it under the same block identifier, calculate the local abscissa difference and local ordinate difference between the location record to be inspected and the previous location chain record, and the local abscissa difference and local ordinate difference between the location record to be inspected and the next location chain record. Merge the location records to be inspected with the same difference direction and the same boundary order into the corresponding location chain. Otherwise, rewrite the location record to be inspected as a disconnected record to obtain the corrected location chain set. S3-5. For each position chain in the correction position chain set, retain the position chains with continuous frame record order and consistent local horizontal coordinate change direction and consistent local vertical coordinate change direction. Delete the position chains other than the first one in ascending order of the sum of the cumulative local horizontal coordinate difference and the cumulative local vertical coordinate difference, and then merge the retained position chains into the same position record to obtain the position correspondence table.

7. The method for detecting quality defects in glass curtain walls based on three-dimensional modeling according to claim 6, characterized in that: S4 includes: S4-1. Read each record of the same position in the position correspondence table, calculate the position mean and texture mean in each record of the same position, and write the position mean and texture mean into the reference point record to obtain the reference point record set; S4-2. Group the reference point record set by plate identifier, connect the reference point records with the same local x-coordinate under the same plate identifier in sequence by local y-coordinate, connect the reference point records with the same local y-coordinate in sequence by local x-coordinate, and form a reference surface by the adjacent connection results, and form a reference edge line by the connection results at the boundary position.

8. The method for detecting quality defects in glass curtain walls based on three-dimensional modeling according to claim 7, characterized in that: S4 further includes: S4-3. For each sampling point, read the reference point record with the same plate identifier that surrounds the local coordinates of the sampling point, solve the surface equation of the corresponding reference surface, calculate the signed perpendicular distance from the sampling point to the reference surface, and write the signed perpendicular distance into the normal difference value. S4-4. For each reference edge line, read the sampling points with the same local coordinates on both sides of the boundary, calculate the coordinate difference between the sampling points on both sides in the normal direction of the reference edge line, write the coordinate difference into the boundary height difference, and then write the normal difference and the boundary height difference into the deviation record to obtain the deviation record set.

9. The method for detecting quality defects in glass curtain walls based on three-dimensional modeling according to claim 8, characterized in that: S5 includes: S5-1. Read each deviation record in the deviation record set, arrange them in the order of plate identifier, boundary identifier, and local coordinates, group the deviation records with empty boundary identifiers, connected local coordinates, and the same sign of normal difference into intra-plate deviation segments, and group the deviation records with the same boundary identifiers, connected local coordinates, and the same sign of boundary height difference into boundary deviation segments, to obtain the deviation segment set. S5-2. Connect the plate deviation segments in the deviation segment set according to the local coordinate order, calculate the local horizontal coordinate increment and local vertical coordinate increment of adjacent deviation records, and write the local horizontal coordinate increment sign and local vertical coordinate increment sign of each adjacent deviation record into the direction code. Plate deviation segments with only one value of the direction code are recorded as crack defects, otherwise plate deviation segments with two or more values ​​of the direction code are recorded as deformation defects. At the same time, each boundary deviation segment is recorded as a misalignment defect, thus obtaining the defect segment set. S5-3. Write each defect segment in the defect segment set into a defect record according to the plate identifier, boundary identifier, and local coordinate range. Merge defect segments with contiguous local coordinates and the same defect category into the same defect record and output the defect record.

10. A glass curtain wall quality defect detection system based on three-dimensional modeling, characterized in that, include: The time-series archiving module is used to perform multi-view scanning and synchronous shooting of the glass curtain wall, and write the sampled content formed at the same acquisition time into the corresponding frame record. The frame records are arranged in ascending order according to the acquisition time to obtain the original frame sequence. The segment generation module is used to perform coordinate unification and adjacent frame splicing on the sampled content in the original frame sequence. It extracts the dividing lines and their intersections in the splicing result, numbers the dividing lines according to the order of projection position, and generates segment records by the closed area enclosed by two adjacent vertical dividing lines and two adjacent horizontal dividing lines. The position merging module reads the sampling points within the plate for each plate record, calculates the perpendicular distance from each sampling point to the four side boundaries, and writes the local coordinates according to the ratio of the left and right perpendicular distances and the ratio of the top and bottom perpendicular distances. Then, it merges the sampling points with the same plate identifier and the closest local coordinates in different frame records into the same position record to obtain the position correspondence table. The benchmark generation module generates a reference surface and reference edge by performing position mean and texture synthesis on the same position record using the position correspondence table as an index. It also writes the normal difference between the sampling point and the reference surface and the boundary height difference between the corresponding sampling points on both sides of the boundary into the deviation record to obtain the deviation record set. The defect output module continuously merges the deviation record set according to the plate identifier, boundary identifier, and local coordinate order to form the plate deviation segment and the boundary deviation segment. It identifies crack defects and deformation defects based on the local coordinate change direction and normal difference distribution of the plate deviation segment, and identifies misalignment defects based on the boundary height difference distribution of the boundary deviation segment, and outputs the defect record.

Images