Semantic segmentation method of concrete arch bridge cracks based on multispectral imaging

Abstract

The present invention relates to the field of image analysis technology, specifically to a method for semantic segmentation of cracks in concrete arch bridges based on multispectral imaging. The method comprises the following steps: collecting multi-time multispectral images using an unmanned aerial vehicle (UAV), segmenting crack regions using K-means on the multispectral images, extracting and sliding normalizing a multi-band spectral vector sequence to generate a principal axis spectral vector, calculating the spectral vector angle and rate of change to mark transition points, screening boundary points by combining directional consistency and gradient scoring, extracting spectral patterns through time-series alignment and Z-score standardization, evaluating discrete fluctuations and correcting anomalies to output a binary segmentation map of concrete arch bridge cracks. In the present invention, a reflectance spectrum sequence is constructed using multi-time multispectral images, crack identification accuracy is improved by combining principal axis features and a sliding window, texture interference is eliminated by using spectral vector angle changes and gradient scoring, and a standardized spectral pattern time series model is generated to enhance boundary location accuracy, reduce noise interference, and support long-term monitoring of structural damage.

Description

Technical Field

[0001] The present invention relates to the technical field of image analysis, and in particular to a semantic segmentation method for concrete arch bridge cracks based on multispectral imaging. Background Art

[0002] The field of image analysis technology involves detecting, identifying, and segmenting objects in images through computer vision and image processing. Its core areas include image preprocessing, feature extraction, image recognition, object detection, and image semantic segmentation. This technology is widely used in fields such as medical image analysis, remote sensing image recognition, traffic monitoring, industrial inspection, and civil engineering structure safety monitoring. In civil engineering structure monitoring, image semantic segmentation can be used to identify surface damage patterns such as cracks and spalling, enabling assessment and management of structural health.

[0003] Traditional semantic segmentation methods for cracks in concrete arch bridges target cracks on the surface of concrete arch bridge structures. By collecting image data from the bridge surface and extracting and classifying image features using shallow convolutional neural networks, these methods achieve segmentation of crack regions. Traditional methods employ single-scale image processing, combined with static threshold segmentation, small convolution kernels to extract texture information, or supervised learning with limited training samples, to identify and annotate crack features in bridge images. These methods primarily rely on image grayscale differences and local edge information to extract crack features, limiting their ability to extract cracks in complex backgrounds.

[0004] Existing technologies rely on shallow convolutional networks to extract single-scale image features. The static threshold segmentation method has poor matching performance for the uneven grayscale distribution on the concrete surface, which can easily lead to missed or false detection of crack edges. The extraction of local edge information is limited by the perception range of small convolution kernels, making it difficult to capture the global contextual association of crack morphology. Supervised learning models are sensitive to complex background noise under limited sample conditions, have insufficient generalization capabilities, and are prone to overfitting during the training process. This ultimately leads to a decrease in the robustness of crack segmentation results under lighting changes or texture interference scenarios, affecting the reliability of practical engineering applications. Summary of the Invention

[0005] In order to solve the technical problems that the existing technology relies on shallow convolutional networks to extract single-scale image features, the static threshold segmentation method has poor matching performance for the uneven grayscale distribution of concrete surfaces, which easily leads to missed or false detection of crack edges, the extraction of local edge information is limited by the perception range of small convolution kernels, and it is difficult to capture the global contextual association of crack morphology, the supervised learning model is sensitive to complex background noise under limited sample conditions, the generalization ability is insufficient, and the training process is prone to overfitting, which ultimately leads to a decrease in the robustness of crack segmentation results under scenes with changing lighting or texture interference, affecting the reliability of practical engineering applications, the embodiment of the present invention provides a semantic segmentation method for concrete arch bridge cracks based on multispectral imaging. The technical solution is as follows:

[0006] On the one hand, a semantic segmentation method for concrete arch bridge cracks based on multispectral imaging is provided, the method comprising:

[0007] S1: Multi-period images are collected using a multispectral imaging device mounted on an unmanned aerial vehicle. The K-means clustering algorithm is used to segment the candidate crack areas. The multi-band reflection intensity values ​​of the pixels are extracted to construct a spectral vector sequence. A parallel sliding window is defined to extract adjacent spectral vector sequences. The vector normalization algorithm is applied to generate the principal axis spectral vector sequence.

[0008] S2: Calculate the angles between adjacent spectral vectors based on the principal axis spectral vector sequence, calculate the rate of change of the angles between adjacent spectral vectors using a first-order difference algorithm, compare the concrete jump threshold range, mark the continuous exceeding points in the sliding window, and obtain the candidate crack boundary jump points;

[0009] S3: Extracting eight-neighborhood spectral data based on the candidate crack boundary jump point, obtaining the candidate point and the eight-neighborhood spectral vector, analyzing the consistency characteristics of the spectral vector direction, and calculating a weighted score in combination with the gradient change rate. The weighted score is compared with the continuity threshold, low-scoring points are eliminated, and the effective crack boundary point is output;

[0010] S4: Perform multi-period image spatial alignment processing based on the effective boundary points of the crack, extract the multi-band reflection intensity sequence of the target pixel, organize the spectrum response trajectory according to the time dimension, apply the Z-score algorithm for normalization processing, and generate a standardized crack spectrum time series.

[0011] As a further solution of the present invention, the method significantly improves the accuracy of crack boundary detection through the coordinated processing of multi-dimensional spectrum vector angle change analysis and direction consistency evaluation, solving the problems of missed detection and blurred segmentation boundaries in existing methods;

[0012] The principal axis spectrum vector sequence is used to enhance the directional consistency in subsequent angle change identification and boundary extraction, forming a self-matching multi-dimensional spectrum pattern detection structure, which significantly improves the accuracy of crack boundary extraction;

[0013] The candidate crack area includes a multi-band reflection intensity spectrum vector and a principal component vector sequence, the candidate crack boundary jump point includes an angle change rate sequence and an over-limit point position mark, the effective crack boundary point includes a directional consistency distribution, a gradient weighted score and a continuity judgment result, and the standardized crack spectrum time series specifically refers to a spatial alignment intensity sequence and a Z-score normalization parameter.

[0014] As a further solution of the present invention, the specific steps of S1 include:

[0015] S101: Use a multispectral imaging device mounted on a drone to collect multi-band images of the target area. Based on the brightness values ​​of the pixels, the K-means clustering algorithm is applied to divide the image area. The variance convergence criterion is used to optimize the pixel classification results to generate pixel category cluster mapping data.

[0016] The variance convergence criterion means that during the clustering process, when the variance change within a cluster between consecutive iterations is lower than a preset threshold, the clustering is determined to have converged and the iteration is terminated;

[0017] The K-means clustering algorithm is an unsupervised clustering method that divides pixels into a preset number of categories based on feature similarity, and optimizes the division results by iteratively updating the category centers;

[0018] S102: Based on the pixel category cluster mapping data, extract pixel points in the crack candidate area, calculate their multi-band reflection intensity values, construct a corresponding spectral vector sequence, set a parallel sliding window to extract the spectral vector set in the window, and generate a spectral vector distribution matrix in the window;

[0019] The window size of the sliding window is set to 3×3 to keep the local information intact, and the sliding step is set to 1 to avoid breaks and missed detections caused by skip sampling;

[0020] S103: calling the spectral vector distribution matrix in the window, normalizing the spectral vectors based on the Euclidean distance formula, adjusting each group of spectral vectors to a unit norm form, selecting the main direction spectral vectors to form a sequence according to the sliding window order, and generating a main axis spectral vector sequence;

[0021] The vector normalization process refers to performing unit conversion on the spectrum vector according to the Euclidean norm to eliminate amplitude differences and retain directional characteristics.

[0022] As a further solution of the present invention, the specific steps of S2 include:

[0023] S201: Based on the principal axis spectral vector sequence, extract the direction cosine parameters of adjacent spectral vectors in sequence, calculate the angle values ​​between the spectral vectors pair by pair according to the cosine calculation formula of the vector angle in three-dimensional space, and record the angles in time series to obtain a spectral vector angle sequence;

[0024] S202: calling the spectral vector angle sequence, using a first-order difference algorithm to perform a difference process on the continuous spectral vector angle values ​​one by one, arranging the difference values ​​according to the sequence index numbers to form a continuous sequence, calculating the spectral vector angle change rate index value, and obtaining the spectral vector angle change rate sequence;

[0025] The spectral vector angle change rate index value represents the change speed of the adjacent spectral vector angles, and is used to characterize the dynamic mutation degree of the concrete spectrum characteristics;

[0026] S203: comparing the concrete jump threshold range based on the spectral vector angle change rate sequence, screening the index numbers whose differential values ​​continuously exceed the upper and lower limits of the jump threshold, counting the exceeding positions according to the number of repeated marks in the sliding window, and generating crack boundary jump candidate points;

[0027] The concrete jump threshold range is the upper and lower limit interval used to determine whether the spectrum vector change is abnormal, and is used to identify the location of potential cracks or structural defects;

[0028] The crack boundary jump candidate point is marked as a crack boundary jump candidate point when the number of points in the window that continuously exceed the concrete jump threshold is greater than or equal to 2.

[0029] As a further solution of the present invention, the spectral vector angle change rate index value is calculated using the formula:

[0030] ;

[0031] in, Representative The index value of the change rate of the spectral vector angle is Represents the first Angle angle, unit is rad, Represents the first Angle angle, unit is rad, Represents the first Angle angle, unit is rad, Represents the first Angle angle, unit is rad, Represents the base reference angle, in rad, selected based on the median angle of the measured spectrum vector. Represents the original weighting coefficient of the spectrum vector angle change The weight of the item is a dimensionless parameter, Represents the length of the sliding original window, which is a dimensionless parameter.

[0032] As a further solution of the present invention, the specific steps of S3 include:

[0033] S301: Based on the candidate crack boundary jump point, extract the corresponding coordinate position index, detect the reflectivity band values ​​of its eight neighborhoods, construct a corresponding multidimensional vector set, arrange the vectors in spatial position order and integrate them into a matrix structure to generate a neighborhood spectrum vector matrix;

[0034] S302: Calling the neighborhood spectral vector matrix, performing a cosine similarity operation on the cosine value of the angle between the neighborhood spectral vector and the center point spectral vector, calculating the mean similarity of adjacent directions, and determining the direction with the largest mean similarity as the main reference direction to obtain a direction consistency ratio value;

[0035] The directional consistency ratio value is used to measure the stability of the spectral similarity of the candidate point in the main reference direction and its adjacent directions, and is an important indicator for identifying the continuity of the crack boundary;

[0036] S303: Call the directional consistency ratio value, combine the grayscale gradient change rate of the candidate point in the main reference direction, and jointly score the two indicator data according to the linear weighting factor. Perform interval comparison based on the generated score value and the set continuity score threshold, eliminate candidate points with score values ​​lower than the threshold, and obtain the effective boundary points of the crack.

[0037] As a further solution of the present invention, the specific steps of S4 include:

[0038] S401: Based on the effective boundary points of the crack, corresponding position indexes in the multi-time period images are extracted, a spatial mapping relationship between the images is established, the spatial registration references of the differentiated time period images are adjusted, the coordinate structure is unified, and a position matching pixel sequence is generated;

[0039] S402: Calling the multi-band reflection information corresponding to each matching point in the position matching pixel sequence, extracting the multi-band reflection values ​​of the pixel at the same position in multiple time periods, arranging the multi-band values ​​in order according to the time dimension, fitting the variation sequence of the differentiated bands in the time scale, calculating the spectral pattern time series fluctuation intensity value, and obtaining a multi-band spectral pattern sequence set;

[0040] The spectral pattern time series fluctuation intensity value refers to the degree of change in multi-spectral reflectance at the same bridge position at different times, which is used to quantitatively reflect structural state fluctuations and assist in crack identification and evolution analysis;

[0041] S403: Call the time series data in the multi-band spectrum sequence set, calculate the mean and standard deviation of each sequence at all time points one by one, use the Z-score algorithm to perform a normalization operation on each reflection intensity value, reconstruct the band sequence into a standardized value set, and obtain the standardized crack spectrum time series.

[0042] As a further solution of the present invention, the intensity value of the temporal fluctuation of the spectral ripple is calculated using the formula:

[0043] ;

[0044] in, Representative The pixel at the The intensity value of the spectral ripple time series fluctuation under each band is a dimensionless parameter. Representative The pixel at the Band and The reflection value at each time point is a dimensionless parameter. Representative The band in The credibility weighted coefficient of the reflection value at each time point is a dimensionless parameter. Representative The band in The image noise interference coefficient at each time point is a dimensionless parameter. Represents the total number of time points and is a dimensionless parameter.

[0045] As a further embodiment of the present invention, the method includes step S5:

[0046] S5: Based on the standardized crack spectrum time series, the discrete degree of the direction of the boundary point spectrum vector and the fluctuation range of the response intensity are evaluated, points exceeding the disturbance threshold are identified, the K-nearest neighbor algorithm is used to search for spectrum vectors with similar directions and correct them, the neighborhood direction mean is calculated to correct abnormal deviation points, and a binary segmentation map of the concrete arch bridge crack is output;

[0047] The concrete arch bridge crack binary segmentation map includes a directional variance index, an intensity fluctuation threshold and a modified directional mean field.

[0048] As a further solution of the present invention, the specific steps of S5 include:

[0049] S501: Based on the standardized crack spectrum time series, calculate the angle offset between the spectrum vector and the adjacent vector, extract the sequence points whose direction change amplitude exceeds the disturbance threshold according to the time series change rate, and generate the direction disturbance fluctuation interval value;

[0050] The disturbance threshold is the minimum angle deviation limit used to determine whether the change in the direction of the spectrum vector is significant;

[0051] The directional disturbance fluctuation interval value refers to the continuous time series interval where the spectrum vector sequence points with directional disturbance are identified in the standardized crack spectrum time series. This interval reflects the sudden change characteristics of the crack morphology in the spatial or temporal dimension.

[0052] S502: calling the spectral vector position in the directional disturbance fluctuation interval value, inputting the K nearest neighbor algorithm, selecting the direction values ​​of the first K points of the directional difference, correcting the deviation direction according to the neighborhood mean, and obtaining the direction correction deviation rate;

[0053] The direction correction deviation rate is a ratio used to measure the degree of correction required for the spectral vector direction under the guidance of the neighborhood mean, and is used to calibrate the direction accuracy of the abnormal disturbance point;

[0054] S503: Adjust the direction of the spectral vector according to the direction correction deviation rate, match the time series intensity matrix, standardize the correction point intensity value and determine the boundary discreteness, filter the points whose response amplitude exceeds the average fluctuation rate interval, and generate a binary segmentation map of the concrete arch bridge crack.

[0055] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0056] Multi-period image data is collected by multispectral imaging equipment, and a multi-band reflection intensity spectrum vector sequence of pixel points is constructed. The main axis feature extraction and sliding window analysis are used to enhance the recognition accuracy of crack candidate areas. The detection of the angle change rate of adjacent spectral vectors and the gradient weighted scoring mechanism are combined to effectively distinguish the complex texture interference on the concrete surface. Through multi-period spatial alignment and standardized spectral pattern time series generation, a continuous monitoring model for the dynamic evolution of cracks is established. This method integrates multi-dimensional analysis of spectral features with spatiotemporal correlation modeling to improve the boundary positioning accuracy of crack segmentation, reduce the impact of environmental noise on the recognition results, and provide highly consistent spatiotemporal data support for long-term trend analysis of structural damage. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 Schematic diagram of the workflow of the present invention. DETAILED DESCRIPTION

[0058] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0059] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0060] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0061] See also Figure 1The embodiment of the present invention provides a semantic segmentation method for concrete arch bridge cracks based on multispectral imaging. The processing flow of the method may include the following steps:

[0062] S1: Multi-period images are collected using a multispectral imaging device mounted on an unmanned aerial vehicle. The K-means clustering algorithm is used to segment the candidate crack areas. The multi-band reflection intensity values ​​of the pixels are extracted to construct a spectral vector sequence. A parallel sliding window is defined to extract adjacent spectral vector sequences. The vector normalization algorithm is applied to generate the principal axis spectral vector sequence.

[0063] S2: Calculate the angles between adjacent spectral vectors based on the principal axis spectral vector sequence, use the first-order difference algorithm to calculate the rate of change of the angles between adjacent spectral vectors, compare it with the concrete jump threshold range, mark the continuous exceeding points in the sliding window, and obtain the candidate points of crack boundary jump;

[0064] S3: Extract eight-neighborhood spectral data based on the candidate points of crack boundary jump, obtain the candidate points and eight-neighborhood spectral vectors, analyze the consistency characteristics of the spectral vector direction, and calculate the weighted score based on the gradient change rate. Compare it with the continuity threshold, eliminate low-scoring points, and output the effective crack boundary point;

[0065] S4: Perform multi-period image spatial alignment based on the effective boundary points of the crack, extract the multi-band reflection intensity sequence of the target pixel, organize the spectrum response trajectory according to the time dimension, apply the Z-score algorithm for normalization, and generate the standardized crack spectrum time series;

[0066] S5: Based on the standardized crack spectrum time series, the discrete degree of the spectral vector direction of the boundary point and the fluctuation range of the response intensity are evaluated. Points exceeding the disturbance threshold are identified. The K-nearest neighbor algorithm is used to search for spectral vectors with similar directions and correct them. The mean value of the neighborhood direction is calculated to correct abnormal deviation points, and a binary segmentation map of the concrete arch bridge crack is output.

[0067] The candidate crack areas include multi-band reflection intensity spectrum vectors and principal component vector sequences. The candidate crack boundary jump points include angle change rate sequences and over-limit point position markers. The effective crack boundary points include directional consistency distribution, gradient weighted scores, and continuity judgment results. The standardized crack spectrum time series specifically refers to the spatially aligned intensity sequence and Z-score normalization parameters. The binary segmentation map of concrete arch bridge cracks includes directional variance indicators, intensity fluctuation thresholds, and corrected directional mean fields.

[0068] Specifically, the steps of S1 are:

[0069] S101: Use a multispectral imaging device mounted on a drone to collect multi-band images of the target area. Based on the brightness values ​​of the pixels, the K-means clustering algorithm is applied to divide the image area. The variance convergence criterion is used to optimize the pixel classification results to generate pixel category cluster mapping data.

[0070] The variance convergence criterion means that in the clustering process, when the variance change within a class between consecutive iterations is lower than a preset threshold, the clustering is judged to have converged and the iteration is terminated;

[0071] The K-means clustering algorithm is an unsupervised clustering method that divides pixels into a preset number of categories based on feature similarity, and optimizes the division results by iteratively updating the category centers.

[0072] Multi-band images of the target area are collected by using a drone equipped with a multispectral imaging device. In this process, the flight path of the drone must first be planned to ensure coverage of the target concrete arch bridge area. The flight altitude is set to 30m, the route overlap is 80% forward and 70% sideways, the acquisition frequency is set to 1Hz, and the multispectral imaging device is set to five channels: red, green, blue, near-infrared, and short-wave infrared. The image resolution of each channel is 1024×1024 pixels. For each pixel, its brightness value in multiple channels is recorded and stored in the form of a five-dimensional vector. After the acquisition is completed, a multi-dimensional pixel data array is constructed. Next, based on the pixel brightness value, a K-means clustering operation needs to be performed. First, the number of clusters is set. In the initialization phase, 5 pixel vectors are randomly selected as the initial class centers. Each class center is a five-dimensional vector, denoted as ,in Represents the average brightness value of the kth class in the jth band. For each pixel vector , and calculate the Euclidean distance between it and the multi-class center in turn:

[0073] ;

[0074] Classify it to the class center with the smallest distance, update the pixel classification label, recalculate the new mean of each class center, and repeat the above iterative process. This process requires the introduction of the variance convergence criterion to determine the termination of the iteration. Specifically: Second and In the iterations, the intra-class variance of multiple categories is recorded separately , calculate the variance change value of each category , if satisfied , then the clustering iteration is terminated, where The preset threshold value needs to be set according to the pixel brightness value range and the image noise level. For example, set , indicating that clustering converges when the variance of the categories in consecutive iterations does not exceed 2.5 unit pixel values. In a certain actual operation, the variance changes of the five categories in the 4th and 5th iterations were [1.8, 2.1, 0.9, 2.3, 1.7] respectively, with the maximum change of 2.3, which is lower than the threshold of 2.5. The convergence condition is met and the clustering process is terminated. The final output is a set of label matrices with the same size as the original image, and each pixel corresponds to a class label. , forming pixel category clustering mapping data.

[0075] Table 1: Multi-band image acquisition parameters

[0076]

[0077] As shown in Table 1, the band data has uniform resolution and quantization accuracy, which is convenient for subsequent vectorization and clustering processing. Combined with the set noise level, a reasonable convergence threshold is determined. , ensuring that the clustering termination conditions are matched, and the clustering output results can be directly used for subsequent screening and analysis of crack candidate areas after they are formed.

[0078] S102: Based on the pixel category cluster mapping data, extract pixel points in the crack candidate area, calculate their multi-band reflection intensity values, construct a corresponding spectral vector sequence, set a parallel sliding window to extract the spectral vector set within the window, and generate a spectral vector distribution matrix within the window;

[0079] The window size of the sliding window is set to 3×3 to keep the local information intact, and the sliding step is set to 1 to avoid breaks and missed detections caused by jump sampling;

[0080] Based on the pixel category cluster mapping data obtained in the previous steps, it is necessary to identify the pixels marked as suspected crack areas one by one, and extract the reflection intensity values ​​of these pixels in the five channels of red light, green light, blue light, near infrared, and short-wave infrared, and construct a five-dimensional spectrum vector representation. For example, if a pixel Under five channels, the corresponding brightness value is [120, 98, 87, 134, 116], then the spectrum vector corresponding to the pixel is , in the entire image, the spectrum vector set in all crack candidate areas is constructed in this way, and then the sliding window parameters are set to perform window operation, and the sliding window size is set to That is, for each central pixel, a local area vector set is constructed from 8 neighboring pixels above, below, left, right and diagonally. The window starts from the upper left corner of the image and slides horizontally with a step size of 1. After completing a line scan, it slides down one pixel unit and repeats the full image scan. Each time it slides, the spectral vectors of the 9 pixels in the current window are recorded as a set , construct the matrix in the original order , the dimension of the spectral vector distribution matrix is , where the rows represent each pixel in the window and the columns represent the five-channel reflectance values. If there are 9 pixels in a window, the corresponding spectrum vectors are as follows:

[0081] , forming the spectral vector distribution matrix.

[0082] During each sliding process, the spectral vector set within the window is extracted and generated in real time. , laying the foundation for subsequent normalization and construction of principal axis spectrum vectors.

[0083] S103: calling the spectral vector distribution matrix in the window, normalizing the spectral vectors based on the Euclidean distance formula, adjusting each group of spectral vectors to the unit norm form, selecting the main direction spectral vectors to form a sequence according to the sliding window order, and generating a main axis spectral vector sequence;

[0084] Vector normalization refers to converting the spectral vector into a unit according to the Euclidean norm to eliminate the amplitude difference and retain the directional characteristics;

[0085] After calling the spectral vector distribution matrix in the window, each spectral vector in the matrix needs to be normalized one by one. Specifically, for each vector Compute its Euclidean norm:

[0086] ;

[0087] Then normalize its components, that is:

[0088] ;

[0089] For example, for the spectral vector , whose norm is:

[0090] ;

[0091] Then its normalized vector is ,right The same calculation is performed on the 9 spectral vectors to form a normalized spectral vector matrix In the normalized matrix, according to the original pixel arrangement of the sliding window, the three spectral vectors on the main diagonal corresponding to the image scanning direction (such as from the upper left to the lower right) are selected as the main direction spectral vectors. For example, the center pixel of the window is , then the corresponding main direction spectrum vector sequence is , and are combined in order to form a principal axis spectrum vector sequence. If the three vectors are:

[0092] ;

[0093] ;

[0094] ;

[0095] Then the main axis spectrum vector sequence is in the form of:

[0096] ;

[0097] ;

[0098] ;

[0099] This sequence is the key input data for subsequent crack feature extraction.

[0100] During this normalization process, the Euclidean norm calculation involves only the square root of the sum of squares of a vector dimension of 5, without any abstract functions or high-order processing. Furthermore, the selection of the principal axis spectral vector is based entirely on the geometric position of pixels within a fixed window, eliminating randomness or complex strategies. Through this process, the spectral vectors of the suspected crack region are normalized window by window and constructed into a sequence of principal axis spectral vectors, encoding the spatial-spectral information structure.

[0101] Specifically, the steps of S2 are:

[0102] S201: Based on the main axis spectrum vector sequence, the direction cosine parameters of adjacent spectrum vectors are extracted in sequence. According to the cosine calculation formula of the vector angle in three-dimensional space, the angle values ​​between the spectrum vectors are calculated pair by pair, and the angles are recorded in time series to obtain a spectrum vector angle sequence;

[0103] Based on the main axis spectrum vector sequence, first extract adjacent spectrum vector pairs in sequence. The current sequence is:

[0104]

[0105]

[0106]

[0107] Combine vector pairs, for example, and As the first pair, calculate its direction cosine, that is, use the cosine formula of the three-dimensional space vector angle:

[0108] ;

[0109] right and The dot product is:

[0110] ;

[0111] Calculate the vector modulus separately:

[0112] ;

[0113] ;

[0114] Substitute the formula to calculate the cosine value: ;

[0115] This gives the angle: ;

[0116] Repeat the process to calculate and The angle between , and finally the spectral vector angle sequence is obtained as , and record the sequence under the time series index, such as between the 1st frame and the 2nd frame , between the 2nd and 3rd frames is Etc., the result is the spectral vector angle sequence.

[0117] S202: calling the spectrum vector angle sequence, using a first-order difference algorithm, performing a difference process on the continuous spectrum vector angle values ​​one by one, arranging the difference values ​​according to the sequence index number to form a continuous sequence, calculating the spectrum vector angle change rate index value, and obtaining the spectrum vector angle change rate sequence;

[0118] The index value of the spectrum vector angle change rate indicates the change speed of the adjacent spectrum vector angles, which is used to characterize the dynamic mutation degree of the concrete spectrum characteristics.

[0119] Call the above obtained spectral vector angle sequence , using the first-order difference method, perform item-by-item difference on the angle value sequence, that is, calculate:

[0120] ;

[0121] As the previous step has If we further calculate , then the difference value is:

[0122] ;

[0123] Set the reference angle , take the sliding window length , the original weighting coefficient is set to , calculate the index value of the spectrum vector angle change rate, using the formula:

[0124] ;

[0125] in, Representative The index value of the change rate of the spectral vector angle is Represents the first Angle angle, unit is rad, Represents the first Angle angle, unit is rad, Represents the first Angle angle, unit is rad, Represents the first Angle angle, unit is rad, Represents the base reference angle, in rad, selected based on the median angle of the measured spectrum vector. Represents the original weighting coefficient of the spectrum vector angle change The weight of the item is a dimensionless parameter, Represents the length of the sliding original window, which is a dimensionless parameter.

[0126] Substitute the parameters and calculate:

[0127] ;

[0128] The step-by-step calculation is as follows:

[0129] , ;

[0130] ;

[0131] final:

[0132] ;

[0133] This value is the index value of the spectrum vector angle change rate at the first position. Continuously performing the above operation can obtain a complete change rate sequence.

[0134] S203: Based on the spectral vector angle change rate sequence, the concrete jump threshold range is compared, and index numbers whose differential values ​​continuously exceed the upper and lower limits of the jump threshold are screened. The exceeding positions are counted according to the number of repeated marks in the sliding window to generate candidate crack boundary jump points;

[0135] The concrete jump threshold range is the upper and lower limit interval used to determine whether the spectrum vector change is abnormal, and is used to identify the location of potential cracks or structural defects;

[0136] A crack boundary jump candidate point is marked as a crack boundary jump candidate point when the number of points in the window that continuously exceed the concrete jump threshold is greater than or equal to 2;

[0137] According to the obtained change rate series , define the upper and lower limits of the crack jump threshold, for example, set the upper limit to 0.035 and the lower limit to 0.005, and take the change rate at the index position Exceeding the upper limit of 0.035, mark index 1 as the limit point; further in the sliding window length In the sequence, determine whether there are more than 2 consecutive over-limit values. If three consecutive items in are greater than 0.035, each index is marked and recorded as a jump candidate point, indicating that there is a crack boundary near the point, as shown in Table 2.

[0138] Table 2: Example table of transition threshold determination

[0139]

[0140] As shown in Table 2, when the change rates of three consecutive points in the sequence all exceed the preset upper threshold of 0.035, the three points are uniformly marked as jump candidate points. This judgment mechanism supports the automatic identification of crack boundary changes in concrete arch bridges.

[0141] The benefit of the formula is that by introducing a weighted calculation of the rate of change within the original sliding window, the disturbance of the previous angle change on the current position is quantified and controlled, and the influence of the absolute angle amplitude is eliminated through normalization processing, thereby enhancing the detection accuracy of continuous abnormal angle changes. This mechanism combines the actual angle sequence fluctuation characteristics to achieve precise identification of the crack boundary area.

[0142] Specifically, the steps of S3 are:

[0143] S301: Based on the candidate crack boundary jump point, the corresponding coordinate position index is extracted, the reflectivity band values ​​of the eight neighborhoods are detected, and the corresponding multidimensional vector set is constructed. The vectors are arranged in spatial position order and integrated into a matrix structure to generate a neighborhood spectrum vector matrix;

[0144] Based on the crack boundary jump candidate points, the coordinate index of the candidate points in the image is first extracted. For example, in a remote sensing image, there are 15 jump points located at row and column positions (120, 87), (121, 88), (122, 89), etc. The row and column indexes of the jump points in the two-dimensional space are calibrated in turn. Then, the corresponding reflectivity band value is extracted in the eight neighborhoods of each candidate point. The processed remote sensing data has multiple bands (such as red, green, blue, near infrared, short-wave infrared, etc.). Assuming that the multispectral image currently used has 5 bands, a neighborhood of each jump point includes 9 pixels in total, and each pixel has 5 band values, which constitute a total of The spectral vector of the dimension is further organized into a vector with the jump point as the center, and the vectors are arranged in the order of the relative positions of the adjacent points and the center point, that is, upper left, upper, upper right, left, middle, right, lower left, lower, and lower right. Repeat the above process for candidate points to form multiple neighborhood spectrum vector matrices. For example, if the number of candidate points is 15, the shape of the final generated matrix set is , take the following data as an example:

[0145] Table 3: Sample table of neighborhood reflectance bands (unit: reflectance value × 1000)

[0146]

[0147] As shown in Table 3, by extracting and arranging the reflectance of each pixel in the multispectral band, a neighborhood spectrum vector matrix is ​​constructed to provide basic data for the next step of directional analysis.

[0148] S302: Calling the neighborhood spectral vector matrix, performing a cosine similarity operation on the cosine value of the angle between the neighborhood spectral vector and the center point spectral vector, calculating the mean similarity of adjacent directions, and determining the direction with the largest mean similarity as the main reference direction to obtain a direction consistency ratio value;

[0149] The directional consistency ratio value is used to measure the stability of the spectral similarity of the candidate point in the main reference direction and its adjacent directions. It is an important indicator for identifying the continuity of the crack boundary.

[0150] After calling the neighborhood spectrum vector matrix, the cosine value of the angle between each non-center vector in the matrix and the center point spectrum vector is calculated respectively. When executing, the five-dimensional reflectivity vector of the center point is first obtained. , and then select a neighborhood point vector , through the formula:

[0151] ;

[0152] Calculate each of the neighborhood points, for example, for the center point:

[0153] ;

[0154] One of its neighboring points is: ,but:

[0155] The numerator is:

[0156] ;

[0157] The denominator is:

[0158] ;

[0159] ;

[0160] but: ;

[0161] The cosine value of each point in the eight neighborhoods is calculated to obtain the similarity values ​​in eight directions. The symmetric neighborhoods in four directions (horizontal, vertical, left diagonal, and right diagonal) are averaged and the set of directions with the highest average values ​​is selected. If the average cosine value of the two symmetric neighbors in the direction is the highest, such as the average value of the (up + down) direction is 0.954, then this direction is set as the main reference direction. The directional consistency ratio is represented by the maximum difference between the mean of the main direction and the mean of the other three directions. For example, if the mean of the main direction is 0.954 and the maximum value in the other directions is 0.910, then the ratio is , this value will be used in the next step of scoring calculation.

[0162] S303: Calling the directional consistency ratio value, combining the grayscale gradient change rate of the candidate point in the main reference direction, and jointly scoring the two indicator data according to the linear weighting factor, performing interval comparison based on the generated score value and the set continuity score threshold, eliminating the candidate points with scores below the threshold, and obtaining the effective boundary points of the crack;

[0163] After calling the direction consistency ratio value, continue to obtain the grayscale gradient change rate of the candidate point in the main reference direction. When executing, the grayscale values ​​of the upper and lower directions of the candidate point are extracted as 135 and 115 respectively, then the grayscale difference in this direction is , normalized to the gray value of the center point 125, the gradient change rate is , set the linear weighting coefficient 、 , linearly combine the directional consistency ratio value R and the gray gradient change rate G, that is, the score value is:

[0164] ;

[0165] The default continuity scoring threshold is set to 0.08. If the score is 0.0904 > 0.08, the candidate point is retained. If the score is below 0.08, it is eliminated. The final set of effective crack boundary points is the set of points with scores above the threshold. This scoring threshold is selected based on experimental analysis and is set at the 40th percentile based on the distribution of candidate point scores. For example, if the sixth value of the sorted candidate point score sequence is 0.08 (for a total of 15 candidate points), the threshold is set to 0.08. This approach avoids the use of fuzzy thresholding and ensures stable and repeatable candidate point screening.

[0166] Specifically, the steps of S4 are:

[0167] S401: Based on the effective boundary points of the crack, the corresponding position indexes in the multi-time period images are extracted, the spatial mapping relationship between the images is established, the spatial registration reference of the differentiated time period images is adjusted, the coordinate structure is unified, and a position matching pixel sequence is generated;

[0168] Based on the effective boundary points of the crack, the coordinate position of the crack target in the multi-time remote sensing image is first obtained, and the pixel coordinate group of the crack boundary point is extracted point by point using a single image as the reference image. Then, the initial matching index corresponding to the reference image is found in the remaining time period images. By comparing the geometric coordinate relationship and pixel grayscale features under the same geographical location, the coordinate matching operation is performed. For matching points with position deviations, a comparison index set is constructed point by point according to the three parameters of pixel grayscale value, edge gradient direction and main direction vector, and a three-value difference comparison operation is performed, that is, the difference item of the target pixel grayscale value and the reference pixel grayscale value, the angle value of the edge gradient direction and the cosine angle value of the main direction vector are calculated respectively. When all three indicators meet the set threshold conditions, it is judged as a successful matching point; if there is a pixel point with a deviation exceeding the threshold interval , a 3×3 pixel window is constructed with this point as the center, and the matching operation of the above three indicators is repeated for the pixels in the window. The pixel closest to the reference image indicator is selected as the mapping point in the window, its coordinates are recorded, and spatial translation and scale affine transformation are performed on it to further reduce the matching error; then a coordinate transformation matrix is ​​constructed with all the mapping points, and the time period images are respectively aligned to the reference image coordinate system to form a unified coordinate structure; on this basis, the pixel coordinates of each crack boundary point in the reference image are used as indexes, and the pixel values ​​at the same position in the registered image in the corresponding time period are sequentially obtained to construct a position matching pixel sequence across time periods.

[0169] In this process, the thresholds of the three indicators are set as follows: the grayscale difference limit does not exceed 15, the edge gradient angle limit does not exceed 8°, and the main direction vector cosine value should be greater than 0.92. If any of the above is exceeded, it is judged as a matching failure. For example, the coordinates of a crack boundary point in the reference image A are (105, 88), and the pixel value difference is 12, the gradient angle is 6°, and the main direction vector cosine is 0.95 at this index position in the target image B. All three meet the thresholds and are judged to be a successful match. The registration coordinates are recorded. Otherwise, the window search operation is entered. The mapping relationship is established through coordinate correction and matrix transformation, and finally the spatial registration results and matching pixel sequences of the crack area in the multi-time period image are formed.

[0170] S402: Calling the multi-band reflection information corresponding to each matching point in the position matching pixel sequence, extracting the multi-band reflection values ​​of the pixel at the same position in multiple time periods, arranging the multi-band values ​​in order according to the time dimension, fitting the variation sequence of the differentiated bands in the time scale, calculating the spectral pattern time series fluctuation intensity value, and obtaining a multi-band spectral pattern sequence set;

[0171] The spectral pattern time series fluctuation intensity value refers to the degree of change in multi-spectral reflectance at the same bridge position at different times. It is used to quantitatively reflect the fluctuation of the structural state and assist in crack identification and evolution analysis.

[0172] Call the multi-band reflection information corresponding to each matching point in the position matching pixel sequence, extract the reflection value data of the time point under the band point by point, and use the two-dimensional data structure Indicates the Position in Band and The reflection value at each moment, and the corresponding reflection value credibility weighted coefficient are obtained at the same time Interference coefficient with noise , then extract the reflection value in the time series in sequence according to the band dimension, form a time series reflection array, and calculate the change characteristics of the sequence in the time dimension; in the specific operation, first calculate the average of each sequence to obtain the average reflection value of the pixel in the band in the whole time period, and then perform weighted calculation on the actual reflection value at each time point, that is, to add it to the credibility weight of the corresponding time point. Multiply, then divide by the noise interference coefficient plus 1, then subtract the average value of the entire period, then sum the difference over the entire period and divide by Finally, take its absolute value to obtain the spectral ripple temporal fluctuation intensity value of the pixel in this band, and calculate the spectral ripple temporal fluctuation intensity value using the formula:

[0173] ;

[0174] in, Representative The pixel at the The intensity value of the spectral ripple time series fluctuation under each band is a dimensionless parameter. Representative The pixel at the Band and The reflection value at each time point is a dimensionless parameter. Representative The band in The credibility weighted coefficient of the reflection value at each time point is a dimensionless parameter. Representative The band in The image noise interference coefficient at each time point is a dimensionless parameter. Represents the total number of time points and is a dimensionless parameter.

[0175] Set T=5, the reflection value sequence of a pixel in band 2 is [0.12, 0.18, 0.14, 0.17, 0.13], the corresponding weighting coefficient is [0.92, 0.88, 0.95, 0.90, 0.91], and the noise interference coefficient is [0.10, 0.20, 0.15, 0.10, 0.18]. The average reflection value is: (0.12+0.18+0.14+0.17+0.13) / 5=0.148,

[0176] Calculate each term in turn:

[0177] ;

[0178] ;

[0179] ;

[0180] ;

[0181] ;

[0182] The sum is: , substituting into the formula we get:

[0183] ;

[0184] The results show that the spectral fluctuation amplitude of this pixel in the second band over the five time points is 0.0682, indicating a low intensity of reflective variation. To assess the relative significance of this value, the benchmark interval is set as [0, 0.05] for low fluctuation, [0.05, 0.10] for medium fluctuation, [0.10, 0.20] for high fluctuation, and fluctuations exceeding 0.20 for abnormal fluctuation. Therefore, 0.0682 is in the medium fluctuation range, corresponding to a moderate trend in the texture characteristics of the crack region.

[0185] S403: Calling the time series data in the multi-band spectrum sequence set, calculating the mean and standard deviation of each sequence at all time points, performing a normalization operation on each reflection intensity value using the Z-score algorithm, reconstructing the band sequence into a standardized value set, and obtaining a standardized crack spectrum time series;

[0186] Call the time series data in the multi-band spectral sequence set, perform mean and standard deviation calculation operations on each band sequence, record the time average and variance values ​​of each pixel point in a specific band, and perform Z-score normalization operation, that is, perform a transformation operation on each reflection value in the sequence by subtracting the mean and then dividing it by the standard deviation to form a standardized numerical sequence. The mean is obtained by taking the arithmetic mean of the reflection value sequence, and the standard deviation is obtained by calculating the square of the difference between each value and the mean, and then taking the square root of the average.

[0187] First For example, the reflectance value sequence of a pixel point in the third band is [0.21, 0.25, 0.23, 0.22, 0.24]. First, calculate the mean: (0.21+0.25+0.23+0.22+0.24) / 5=0.23, and calculate the standard deviation:

[0188] ;

[0189] Perform a Z-score transformation on each value:

[0190] ;

[0191] The final normalized sequence is [-1.42, 1.42, 0, -0.71, 0.71], which represents the relative change trend of the pixel under differentiated time.

[0192] Table 4: Conversion table of spectral ripple value and standard deviation

[0193]

[0194] As shown in Table 4, the original reflectance sequence of a certain pixel point in a specific band and its Z-score normalized conversion results are listed. The normalization operation eliminates the influence of absolute value on the result fluctuation, making it suitable for subsequent unified processing.

[0195] Specifically, the steps of S5 are:

[0196] S501: Based on the normalized crack spectrum time series, the angle offset between the spectrum vector and the adjacent vector is calculated, and the sequence points whose direction change amplitude exceeds the disturbance threshold are extracted according to the time series change rate to generate the direction disturbance fluctuation interval value;

[0197] The disturbance threshold is the minimum angle deviation limit used to determine whether the change in the direction of the spectrum vector is significant;

[0198] The directional disturbance fluctuation interval value refers to the continuous time series interval where the spectrum vector sequence points with directional disturbance are identified in the standardized crack spectrum time series. This interval reflects the sudden change characteristics of the crack morphology in the spatial or temporal dimension.

[0199] Based on the standardized crack spectrum time series, the spectrum vector can be recorded as a set of feature vectors recorded on continuous time frames, which can be recorded as , where each vector Representative The directional features extracted from the crack image in the frequency domain at the frame moment, such as the edge gradient direction, can be calculated by taking the energy value vector reflected by the main direction of each unit pixel area in the image in the Fourier domain as input, and then performing the calculation on the adjacent spectral vectors. and Perform angle offset calculation using the vector angle calculation formula:

[0200] ;

[0201] Obtain the angle difference sequence between multiple frames , and calculate the rate of change of the angle with time series , and then filter out The sequence points, where the disturbance threshold is a set value used to eliminate small-scale angle disturbances caused by random noise, such as the angle change in the stable section of the crack propagation path. , you can set ,like , then the point sequence is marked as a disturbance point. Finally, by traversing the sequence points, extracting the continuous disturbance point segments and calculating their length, angle fluctuation mean and maximum values, a list of directional disturbance fluctuation interval values ​​is formed. , each Indicates a continuous area of ​​sudden changes in direction. For example, if the angle between the 15th and 23rd frames in a crack monitoring sequence of a concrete arch bridge fluctuates continuously and significantly, the maximum angle is , corresponding to the interval It is recorded as a directional disturbance fluctuation interval value, and then enters the direction correction calculation stage.

[0202] S502: Call the spectral vector position in the direction disturbance fluctuation interval value, input the K nearest neighbor algorithm, select the direction difference before the K points to count the direction value, correct the deviation direction according to the neighborhood mean, and obtain the direction correction deviation rate;

[0203] The direction correction deviation rate is used to measure the ratio of the degree of correction required for the spectral vector direction under the guidance of the neighborhood mean, and is used to calibrate the direction accuracy of the abnormal disturbance point;

[0204] Call the disturbance fluctuation interval value obtained in the previous stage , for each interval , get its internal spectrum vector set The spectral vector position index value in arrive , for each spectral vector Select the adjacent time series in this interval The spectral vectors constitute a neighborhood set , and and the vector in the neighborhood Calculate direction angle , the directional difference is defined as The average value of:

[0205] ;

[0206] when There is a significant deviation from the neighborhood mean direction (e.g. ), direction correction is performed, that is, the current vector direction is replaced by the neighborhood direction mean, and the new direction is defined as:

[0207] ;

[0208] Simultaneously record the direction correction deviation rate . Let’s take a specific example to illustrate that if the first The direction of the vector is , the mean direction of the 5 spectral vectors in the neighborhood is , then , calculate the direction correction deviation rate as:

[0209] ;

[0210] when Exceeds the set direction deviation correction threshold (For example, when it is set to 10%), confirm that the direction replacement is required. The neighborhood correction calculation results are recorded as follows:

[0211] Table 5: Direction Correction Deviation Rate Table

[0212]

[0213] As shown in Table 5, the correction deviation rate of the data in the first row is significantly higher than the threshold, and the direction replacement operation is performed. The second and third rows have low-amplitude deviations and are only recorded without replacement.

[0214] S503: Adjust the direction of the spectrum vector according to the direction correction deviation rate, match the time series intensity matrix, standardize the correction point intensity value and determine the boundary dispersion, filter the points whose response amplitude exceeds the average fluctuation rate interval, and generate a binary segmentation map of the concrete arch bridge crack;

[0215] According to the above-mentioned corrected spectral vector The direction is matched with the original spectrum vector sequence, and the corrected vector sequence is used to construct the time series intensity matrix , where each row represents the correction direction of a time frame and the columns are the frequency domain intensity values, Represents the original intensity of the jth frequency point in the i-th frame in the matrix M, which is recorded as , for the matrix The intensity of multiple units is normalized, that is, each column is normalized to Interval, perform the normalization formula:

[0216] ;

[0217] Then, the boundary discreteness detection is performed on the time frame, and the difference between the maximum and minimum intensity values ​​in each frame is defined as the discreteness , to set the threshold Determine whether the boundary strength is discrete, e.g. ,when The frame is recorded as a crack-significant frame, and then the points where the response amplitude exceeds the average fluctuation rate are extracted. The fluctuation rate is defined as the absolute value of the mean difference in the intensity of consecutive frames. ,like Exceeding the average fluctuation If the number of times the standard deviation is greater than 1, it is determined to be a strong response point. Finally, in the image space, the significant response points of the frame are set to 1, and the rest are set to 0 to form the final crack binary segmentation map. The maximum value of the normalized intensity in the 10th frame is 0.94, and the minimum value is 0.22. , identified as a boundary discrete frame, and if , , and the fluctuation mean is 0.08 and the standard deviation is 0.06, then , the critical strong response point condition is met, the frame is judged as a crack response frame, and the corresponding pixel in the image is set to 1 for outputting a binary image.

[0218] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A semantic segmentation method for concrete arch bridge cracks based on multispectral imaging, characterized by: The following steps are involved: S1: Multi-period images are collected using a multispectral imaging device mounted on an unmanned aerial vehicle. The K-means clustering algorithm is used to segment the candidate crack areas. The multi-band reflection intensity values ​​of the pixels are extracted to construct a spectral vector sequence. A parallel sliding window is defined to extract adjacent spectral vector sequences. The vector normalization algorithm is applied to generate the principal axis spectral vector sequence. S2: Calculate the angles between adjacent spectral vectors based on the principal axis spectral vector sequence, calculate the rate of change of the angles between adjacent spectral vectors using a first-order difference algorithm, compare the concrete jump threshold range, mark the continuous exceeding points in the sliding window, and obtain the candidate crack boundary jump points; S3: Extracting eight-neighborhood spectral data based on the candidate crack boundary jump point, obtaining the candidate point and the eight-neighborhood spectral vector, analyzing the consistency characteristics of the spectral vector direction, and calculating a weighted score in combination with the gradient change rate. The weighted score is compared with the continuity threshold, low-scoring points are eliminated, and the effective crack boundary point is output; S4: Perform multi-period image spatial alignment processing based on the effective boundary points of the crack, extract the multi-band reflection intensity sequence of the target pixel, organize the spectrum response trajectory according to the time dimension, apply the Z-score algorithm for normalization processing, and generate a standardized crack spectrum time series.

2. The method for semantic segmentation of concrete arch bridge cracks based on multispectral imaging according to claim 1, characterized in that: The candidate crack area includes a multi-band reflection intensity spectrum vector and a principal component vector sequence, the candidate crack boundary jump point includes an angle change rate sequence and an over-limit point position mark, the effective crack boundary point includes a directional consistency distribution, a gradient weighted score and a continuity judgment result, and the standardized crack spectrum time series specifically refers to a spatial alignment intensity sequence and a Z-score normalization parameter.

3. The method for semantic segmentation of concrete arch bridge cracks based on multispectral imaging according to claim 1, characterized in that: The specific steps of S1 include: S101: Use a multispectral imaging device mounted on a drone to collect multi-band images of the target area. Based on the brightness values ​​of the pixels, the K-means clustering algorithm is applied to divide the image area. The variance convergence criterion is used to optimize the pixel classification results to generate pixel category cluster mapping data. S102: Based on the pixel category cluster mapping data, extract pixel points in the crack candidate area, calculate their multi-band reflection intensity values, construct a corresponding spectral vector sequence, set a parallel sliding window to extract the spectral vector set in the window, and generate a spectral vector distribution matrix in the window; S103: calling the spectral vector distribution matrix in the window, normalizing the spectral vectors based on the Euclidean distance formula, adjusting each group of spectral vectors to the unit norm form, selecting the main direction spectral vectors to form a sequence in the sliding window order, and generating a main axis spectral vector sequence.

4. The method for semantic segmentation of concrete arch bridge cracks based on multispectral imaging according to claim 3, characterized in that: The specific steps of S2 include: S201: Based on the principal axis spectral vector sequence, extract the direction cosine parameters of adjacent spectral vectors in sequence, calculate the angle values ​​between the spectral vectors pair by pair according to the cosine calculation formula of the vector angle in three-dimensional space, and record the angles in time series to obtain a spectral vector angle sequence; S202: calling the spectral vector angle sequence, using a first-order difference algorithm to perform a difference process on the continuous spectral vector angle values ​​one by one, arranging the difference values ​​according to the sequence index numbers to form a continuous sequence, calculating the spectral vector angle change rate index value, and obtaining the spectral vector angle change rate sequence; S203: Based on the spectral vector angle change rate sequence, the concrete jump threshold range is compared, index numbers whose differential values ​​continuously exceed the upper and lower limits of the jump threshold are screened, and the exceeding positions are counted according to the number of repeated marks in the sliding window to generate crack boundary jump candidate points.

5. The method for semantic segmentation of concrete arch bridge cracks based on multispectral imaging according to claim 4, characterized in that: The spectral vector angle change rate index value is calculated using the formula: ； in, Representative The index value of the change rate of the spectral vector angle is Represents the first Angle angle, unit is rad, Represents the first Angle angle, unit is rad, Represents the first Angle angle, unit is rad, Represents the first Angle angle, unit is rad, Represents the base reference angle, in rad, selected based on the median angle of the measured spectrum vector. Represents the original weighting coefficient of the spectrum vector angle change The weight of the item is a dimensionless parameter, Represents the length of the sliding original window, which is a dimensionless parameter.

6. The method for semantic segmentation of concrete arch bridge cracks based on multispectral imaging according to claim 4, characterized in that: The specific steps of S3 include: S301: Based on the candidate crack boundary jump point, extract the corresponding coordinate position index, detect the reflectivity band values ​​of its eight neighborhoods, construct a corresponding multidimensional vector set, arrange the vectors in spatial position order and integrate them into a matrix structure to generate a neighborhood spectrum vector matrix; S302: Calling the neighborhood spectral vector matrix, performing a cosine similarity operation on the cosine value of the angle between the neighborhood spectral vector and the center point spectral vector, calculating the mean similarity of adjacent directions, and determining the direction with the largest mean similarity as the main reference direction to obtain a direction consistency ratio value; S303: Call the directional consistency ratio value, combine the grayscale gradient change rate of the candidate point in the main reference direction, and jointly score the two indicator data according to the linear weighting factor. Perform interval comparison based on the generated score value and the set continuity score threshold, eliminate candidate points with score values ​​lower than the threshold, and obtain the effective boundary points of the crack.

7. The method for semantic segmentation of concrete arch bridge cracks based on multispectral imaging according to claim 6, characterized in that: The specific steps of S4 include: S401: Based on the effective boundary points of the crack, corresponding position indexes in the multi-time period images are extracted, a spatial mapping relationship between the images is established, the spatial registration references of the differentiated time period images are adjusted, the coordinate structure is unified, and a position matching pixel sequence is generated; S402: Calling the multi-band reflection information corresponding to each matching point in the position matching pixel sequence, extracting the multi-band reflection values ​​of the pixel at the same position in multiple time periods, arranging the multi-band values ​​in order according to the time dimension, fitting the variation sequence of the differentiated bands in the time scale, calculating the spectral pattern time series fluctuation intensity value, and obtaining a multi-band spectral pattern sequence set; S403: Call the time series data in the multi-band spectrum sequence set, calculate the mean and standard deviation of each sequence at all time points one by one, use the Z-score algorithm to perform a normalization operation on each reflection intensity value, reconstruct the band sequence into a standardized value set, and obtain the standardized crack spectrum time series.

8. The method for semantic segmentation of concrete arch bridge cracks based on multispectral imaging according to claim 7, characterized in that: The intensity value of the temporal fluctuation of the spectral pattern is calculated using the formula: ； in, Representative The pixel at the The intensity value of the spectral ripple time series fluctuation under each band is a dimensionless parameter. Representative The pixel at the Band and The reflection value at each time point is a dimensionless parameter. Representative The band in The credibility weighted coefficient of the reflection value at each time point is a dimensionless parameter. Representative The band in The image noise interference coefficient at each time point is a dimensionless parameter. Represents the total number of time points and is a dimensionless parameter.

9. The method for semantic segmentation of concrete arch bridge cracks based on multispectral imaging according to claim 1, characterized in that: The method comprises step S5: S5: Based on the standardized crack spectrum time series, the discrete degree of the direction of the boundary point spectrum vector and the fluctuation range of the response intensity are evaluated, points exceeding the disturbance threshold are identified, the K-nearest neighbor algorithm is used to search for spectrum vectors with similar directions and correct them, the neighborhood direction mean is calculated to correct abnormal deviation points, and a binary segmentation map of the concrete arch bridge crack is output; The concrete arch bridge crack binary segmentation map includes a directional variance index, an intensity fluctuation threshold and a modified directional mean field.

10. The method for semantic segmentation of concrete arch bridge cracks based on multispectral imaging according to claim 9, characterized in that: The specific steps of S5 include: S501: Based on the standardized crack spectrum time series, calculate the angle offset between the spectrum vector and the adjacent vector, extract the sequence points whose direction change amplitude exceeds the disturbance threshold according to the time series change rate, and generate the direction disturbance fluctuation interval value; S502: calling the spectral vector position in the directional disturbance fluctuation interval value, inputting the K nearest neighbor algorithm, selecting the direction values ​​of the first K points of the directional difference, correcting the deviation direction according to the neighborhood mean, and obtaining the direction correction deviation rate; S503: Adjust the direction of the spectral vector according to the direction correction deviation rate, match the time series intensity matrix, standardize the correction point intensity value and determine the boundary discreteness, filter the points whose response amplitude exceeds the average fluctuation rate interval, and generate a binary segmentation map of the concrete arch bridge crack.

Images