A road crack unmanned aerial vehicle detection surveying and mapping method and system

Abstract

The embodiment of the application provides a kind of road crack unmanned aerial vehicle detection surveying and mapping method and system, belong to image surveying and mapping technical field.The method includes: based on unmanned aerial vehicle collection road surface sequence image, and record corresponding exposure parameter and attitude parameter, obtain the sequence image data for brightness compensation;Based on sequence image data, execute brightness normalization compensation and generate brightness compensation sequence image, construct optical flow vector field in combination with attitude parameter;Based on optical flow vector field, calculate texture smoothness index and execute texture smooth area weight reduction processing, generate crack direction analysis optical flow data for crack direction extraction;Based on crack direction analysis optical flow data, extract crack direction and introduce sub-pixel optical flow differential operator to generate crack depth variation index, form crack morphology data;Based on crack morphology data, execute ground coordinate registration and generate road crack surveying and mapping result.The stability and accuracy of road crack direction identification and space surveying and mapping are improved in the application scheme.

Description

Technical Field

[0001] This invention relates to the field of image mapping technology, and specifically to a method and system for detecting and mapping road cracks using a drone. Background Technology

[0002] Road cracks are a common surface defect and one of the most sensitive damage signals during the long-term service of transportation facilities. Changes in crack orientation are often accompanied by stress transfer within the local material, while changes in depth are frequently related to the weakening of the base structure. Therefore, the industry has always regarded crack morphology as a primary indicator of road health. On-site detection typically relies on ground sensors, manual inspections, or vehicle-mounted imaging equipment, but these methods are easily limited by perspective and coverage, and often fail to obtain stable crack geometry information when encountering complex road structures.

[0003] In recent years, drone imagery has begun to be used in road inspection, improving image acquisition efficiency. However, most solutions still remain within the framework of static image processing, treating each frame as an independent sample and only able to identify the outlines of shallow textures. Subtle changes in cracks are not fully presented in a single frame, especially in asphalt pavement environments with large lighting fluctuations and significant texture interference. Traditional edge detection or thresholding often struggles to distinguish the differences between real cracks and background textures.

[0004] On the other hand, road surface textures exhibit minute displacements over continuous viewing angles, and these displacements contain implicit information about crack geometry changes. Optical flow is a means of characterizing these minute displacements, but using optical flow in road scenes is not straightforward. Illumination jumps alter local pixel brightness, and uncompensated optical flow can easily drift in shadow areas; drone attitudes continuously change during flight, introducing overall offsets into the image sequence; and the rough, granular texture of the asphalt surface closely resembles crack details, further interfering with the stability of optical flow. These combined problems make it difficult for existing techniques to obtain stable optical flow structures from sequential images that can be used for crack geometry analysis.

[0005] Existing research has attempted to incorporate sequence information, but most focus only on image enhancement or trajectory smoothing, failing to truly utilize the directional distribution, amplitude variations, or differential structure of optical flow to infer the direction and depth trends of cracks. The lack of detailed processing for road scenes means these methods still suffer from accuracy fluctuations in actual inspections. In particular, features related to the underlying structure, such as depth variations, are almost impossible to obtain using single-frame methods in most image detection schemes; therefore, relevant data often relies on human experience for judgment.

[0006] For these reasons, the industry has long lacked a sequence analysis method that can systematically handle brightness fluctuations, attitude disturbances, and texture interference in road environments, in order to extract stable crack orientation information from UAV-captured image sequences and further obtain estimates of crack depth changes. Summary of the Invention

[0007] The purpose of this invention is to provide a method and system for detecting and mapping road cracks using unmanned aerial vehicles (UAVs), so as to at least solve the problem that it is difficult to stably obtain information on the direction and depth of cracks under conditions of light fluctuations, attitude disturbances and rough road surface texture in existing road inspections.

[0008] To achieve the above objectives, the first aspect of the present invention provides a method for detecting and mapping road cracks using a UAV. The method includes: acquiring a sequence of road surface images based on the UAV along a preset flight trajectory, and recording the corresponding exposure parameters and attitude parameters to obtain sequence image data for brightness compensation; performing brightness normalization compensation based on the sequence image data to generate a brightness-compensated sequence image, and constructing an optical flow vector field in conjunction with the attitude parameters; calculating a texture stability index based on the optical flow vector field and performing texture stability region weight reduction processing to generate crack direction analysis optical flow data for crack direction extraction; extracting the crack direction based on the crack direction analysis optical flow data and introducing a sub-pixel optical flow differential operator to generate a crack depth change index, forming crack morphology data; wherein, the sub-pixel optical flow differential operator is used to perform sub-pixel reconstruction of the optical flow vector field by interpolation on the basis of the optical flow vector field, and performing differential calculation on the optical flow direction component and amplitude component in the pixel neighborhood to obtain the displacement gradient change of the optical flow vector field; and performing ground coordinate registration based on the crack morphology data to generate road crack mapping results.

[0009] Optionally, based on the UAV acquiring a sequence of road surface images along a preset flight trajectory and recording the corresponding exposure and attitude parameters, a sequence of image data for brightness compensation is obtained, including: acquiring continuous road surface image frames according to a preset image acquisition sequence during the UAV's flight along the preset flight trajectory, and synchronously recording the exposure and attitude parameters at the corresponding frame time when acquiring each road surface image frame; performing time tag pairing processing on the road surface image frames, the exposure parameters, and the attitude parameters based on the acquisition sequence to generate a sequence of image tags for defining the continuity between frames; and aggregating the road surface image frames with the sequence of image tags, the corresponding exposure parameters, and the corresponding attitude parameters to form a sequence of image data for brightness compensation.

[0010] Optionally, performing brightness normalization compensation based on the sequence image data and generating a brightness-compensated sequence image, and constructing an optical flow vector field in conjunction with the attitude parameters, includes: performing brightness shift estimation processing on each road image frame in the sequence image data, generating brightness shift features to represent local brightness changes based on the corresponding exposure parameters; performing brightness normalization compensation processing on the corresponding road image frames based on the brightness shift features, generating a brightness-compensated image frame sequence corresponding to the original acquisition order; constructing initial optical flow data to represent pixel displacement based on the brightness-compensated image frame sequence in the order of adjacent frames, and introducing the corresponding attitude parameters into the initial optical flow data to perform attitude correction processing, generating the corresponding optical flow vector field.

[0011] Optionally, the corresponding attitude parameters are introduced into the initial optical flow data to perform attitude correction processing and generate a corresponding optical flow vector field. This includes: extracting attitude features to characterize the attitude changes of the UAV based on the attitude parameters, and constructing an attitude transformation matrix to reflect the viewpoint differences between adjacent frames based on the attitude features; applying the attitude transformation matrix to the initial optical flow data, performing coordinate system alignment processing on the optical flow vectors corresponding to each pixel in the initial optical flow data, and generating corrected optical flow data to reflect the displacement of road surface texture; and converging the corrected optical flow data according to the sequence frame order to form the optical flow vector field.

[0012] Optionally, based on the optical flow vector field, a texture stability index is calculated and a texture stability region weight reduction processing is performed to generate crack direction analysis optical flow data for crack direction extraction. This includes: extracting the direction component and amplitude component of each optical flow vector based on the optical flow vector corresponding to each pixel in the optical flow vector field, and performing statistical operations on the direction component and amplitude component in the neighborhood of each pixel to generate a texture stability index for characterizing local texture stability; determining the texture stability region to which the corresponding pixel belongs based on the texture stability index, and performing weight adjustment processing on the optical flow vectors in the optical flow vector field that are in the texture stability region according to the texture stability index to generate weighted optical flow data for weakening the influence of the texture stability region; and converging the weighted optical flow data according to the sequence frame order to form crack direction analysis optical flow data for crack direction extraction.

[0013] Optionally, the texture stability region to which the corresponding pixel belongs is determined according to the texture stability index, and the optical flow vectors in the optical flow vector field located in the texture stability region are weighted according to the texture stability index to generate reduced-weighted optical flow data for weakening the influence of the texture stability region. This includes: classifying the texture stability of each pixel in the optical flow vector field according to the texture stability index to determine the texture stability region to which each pixel belongs; mapping the optical flow vectors in the texture stability region to the texture stability index, and performing weight adjustment on the direction and amplitude components of the optical flow vectors according to the texture stability index to generate corresponding reduced-weighted optical flow vectors; and converging the reduced-weighted optical flow vectors according to the sequence frame order to form reduced-weighted optical flow data for crack orientation extraction.

[0014] Optionally, crack orientation is extracted based on the crack orientation analysis optical flow data, and a sub-pixel optical flow differential operator is introduced to generate a crack depth variation index, forming crack morphology data. This includes: extracting optical flow direction components reflecting the local displacement direction distribution based on the optical flow vectors corresponding to each pixel in the crack orientation analysis optical flow data, and performing direction aggregation processing on the optical flow direction components to generate crack orientation features characterizing the linear orientation of the crack; performing corresponding processing on the crack orientation features and the crack orientation analysis optical flow data, and introducing a sub-pixel optical flow differential operator characterizing the local differential gradient into the corresponding processing result, performing differential calculation on the optical flow vectors corresponding to each pixel to generate a crack depth variation index reflecting the crack depth variation; and performing convergence processing on the crack orientation features and the crack depth variation index according to the corresponding frame order to form crack morphology data for three-dimensional reconstruction of road cracks.

[0015] Optionally, performing ground coordinate registration based on the crack morphology data and generating road crack mapping results includes: extracting crack spatial features to characterize the spatial positional relationship between frames based on the crack direction features and crack depth change indicators corresponding to each frame in the crack morphology data; performing correspondence processing between the crack spatial features and attitude parameters used to characterize the flight attitude of the UAV, and constructing a ground coordinate registration relationship to limit the spatial position of the crack based on the correspondence processing results, generating crack registration data in a unified coordinate system; and performing sequence aggregation processing on the crack registration data according to spatial continuity requirements to form road crack mapping results that reflect the direction and depth changes of road cracks.

[0016] Optionally, the method further includes: performing crack segment aggregation processing on the road crack mapping results based on the crack orientation and crack depth changes corresponding to each crack segment in the road crack mapping results, generating crack segment aggregation data to characterize the overall structure of the road cracks; extracting crack path features to describe the overall path continuity of the cracks based on the crack segment aggregation data, and constructing a crack path set to express the spatial distribution of the cracks based on the crack path features; and performing corresponding processing between the crack path set and the road crack mapping results to generate structured output data of road cracks for subsequent road maintenance analysis.

[0017] A second aspect of the present invention provides a road crack detection and mapping system using an unmanned aerial vehicle (UAV). The system includes: an acquisition unit for acquiring a sequence of road surface images based on a UAV along a preset flight trajectory, and recording corresponding exposure parameters and attitude parameters to obtain a sequence of image data for brightness compensation; a compensation unit for performing brightness normalization compensation based on the sequence of image data and generating a brightness-compensated sequence of images, and constructing an optical flow vector field in conjunction with the attitude parameters; a processing unit for calculating a texture stability index based on the optical flow vector field and performing texture stability region weight reduction processing to generate crack direction analysis optical flow data for crack direction extraction; an index generation unit for extracting crack direction based on the crack direction analysis optical flow data and introducing a sub-pixel optical flow differential operator to generate a crack depth change index, forming crack morphology data; wherein the sub-pixel optical flow differential operator is used to perform sub-pixel reconstruction of the optical flow vector field by interpolation, and perform differential calculations on the optical flow direction component and amplitude component in the pixel neighborhood to obtain the displacement gradient change of the optical flow vector field; and a registration unit for performing ground coordinate registration based on the crack morphology data and generating road crack mapping results.

[0018] Through the above technical solution, this invention simultaneously records exposure and attitude parameters during the acquisition phase, providing the sequential images with the context required for subsequent compensation processing. This allows brightness normalization and attitude correction to maintain the continuity of the optical flow structure under conditions of illumination jumps and flight attitude changes. The optical flow vector field after brightness compensation undergoes texture stability analysis and weight reduction processing, effectively eliminating interference from coarse asphalt textures and making the crack-related directional structure more prominent in the optical flow domain. The crack orientation analysis optical flow data constructed based on this provides relatively clear orientation linear characteristics. Furthermore, the introduction of sub-pixel optical flow differential operators amplifies subtle gradient changes in optical flow, enabling the separation of crack depth variations in the image. Finally, the crack morphology data, combined with ground coordinate registration, yields crack mapping results in a unified space, allowing both the orientation continuity and depth trend of cracks to be presented in a spatial form. This provides a structured and more scene-adaptable geometric information foundation for road defect assessment.

[0019] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0020] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings:

[0021] Figure 1 This is a flowchart of the steps of a road crack UAV detection and mapping method provided in one embodiment of the present invention;

[0022] Figure 2 This is a system structure diagram of a road crack UAV detection and mapping system provided in one embodiment of the present invention;

[0023] Figure 3 This is an internal structural diagram of a computer device provided in one embodiment of the present invention. Detailed Implementation

[0024] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0025] like Figure 1 As shown, this invention provides a method for detecting and mapping road cracks using a UAV, the method comprising:

[0026] Step S1: Collect road surface sequence images along the preset flight path using the UAV, and record the corresponding exposure parameters and attitude parameters to obtain sequence image data for brightness compensation.

[0027] Specifically, during the flight of the UAV along a preset flight trajectory, continuous road surface image frames are acquired according to a preset image acquisition sequence, and the exposure parameters and attitude parameters at the corresponding frame time are recorded synchronously when acquiring each road surface image frame; based on the acquisition sequence, time tag pairing processing is performed on the road surface image frames, the exposure parameters and the attitude parameters to generate a sequence image tag for limiting the continuity between frames; the road surface image frames with the sequence image tag, the corresponding exposure parameters and the corresponding attitude parameters are aggregated to form a sequence image data for brightness compensation.

[0028] In this embodiment of the invention, as the UAV moves along a preset flight path, the imaging unit captures road surface texture at a fixed rhythm. Since the flight attitude changes every instant, recording these changes is crucial. According to the pre-set acquisition sequence, road surface image frames are continuously written to the buffer, while exposure parameters and attitude parameters are read simultaneously.

[0029] Before the data enters the post-processing chain, a clear temporal correspondence needs to be established for the three types of data. Here, a time-stamp pairing method is used to bind image frames with concurrent exposure and pose parameters into a single associated unit. This approach is simple but helps ensure that the continuity between adjacent frames is not disrupted during processing. A time stamp is not simply a timestamp, but a sequence identifier used for sorting and association, essentially limiting inter-frame misalignment. Each set of stamps increments as the sequence progresses, giving the entire sequence a clear chronological order.

[0030] After generating the image sequence labels, a convergence process needs to be performed on the acquired road surface image frames and their corresponding exposure and attitude parameters. Convergence here refers to associating and binding the three types of data according to the same sequence number and encapsulating them into a data structure that can be directly read by the brightness compensation module. This action is not for data compression, but to ensure that images, exposure parameters, and attitude parameters under the same frame number can be considered as a coherent input unit. The resulting image sequence data is more stable and better suited for subsequent brightness shift estimation and illumination compensation.

[0031] After the above steps, the final sequence of image data possesses a complete temporal structure and synchronized acquisition attributes. This organization reduces interference from illumination fluctuations for subsequent brightness normalization compensation and provides necessary contextual support for subsequent optical flow calculations.

[0032] Step S2: Perform brightness normalization compensation based on the sequence image data and generate brightness-compensated sequence images, and construct an optical flow vector field by combining the attitude parameters.

[0033] Specifically, brightness shift estimation processing is performed on each road surface image frame in the sequence image data, and brightness shift features are generated to represent local brightness changes based on the corresponding exposure parameters; brightness normalization compensation processing is performed on the corresponding road surface image frames based on the brightness shift features to generate a brightness-compensated image frame sequence corresponding to the original acquisition order; initial optical flow data to represent pixel displacement is constructed based on the brightness-compensated image frame sequence in the order of adjacent frames, and the corresponding attitude parameters are introduced into the initial optical flow data to perform attitude correction processing to generate the corresponding optical flow vector field.

[0034] Furthermore, the corresponding attitude parameters are introduced into the initial optical flow data to perform attitude correction processing and generate the corresponding optical flow vector field. This includes: extracting attitude features to characterize the attitude changes of the UAV based on the attitude parameters, and constructing an attitude transformation matrix to reflect the viewpoint differences between adjacent frames based on the attitude features; applying the attitude transformation matrix to the initial optical flow data, performing coordinate system alignment processing on the optical flow vectors corresponding to each pixel in the initial optical flow data, and generating correction optical flow data to reflect the displacement of road surface texture; and converging the correction optical flow data according to the sequence frame order to form the optical flow vector field.

[0035] In this embodiment of the invention, the processing at this stage mainly focuses on two directions. One direction focuses on brightness normalization, making the brightness performance between different frames more stable; the other direction focuses on attitude correction, so that the optical flow vector is as free as possible from the influence of flight attitude disturbances. The two processing chains eventually converge into a unified optical flow vector field, providing basic data for subsequent crack orientation and depth analysis.

[0036] Brightness normalization typically begins with brightness shift estimation. The drone's perspective changes continuously during flight, and road surface reflection conditions fluctuate with position and time, often resulting in inconsistent overall brightness across different frames. To compensate for these variations, a linear brightness shift model can be constructed based on exposure parameters to correct the original brightness of each pixel. Brightness compensation can be written as:

[0037] ;

[0038] in, Indicates pixel position The original brightness value, This indicates the exposure parameters corresponding to the current image frame. This is a proportionality coefficient used to describe the effect of exposure changes on brightness shift. This represents the pixel brightness value after brightness offset compensation. Through this relationship, operators can perform linear corrections at the pixel level based on exposure parameters, providing clear calculation rules for subsequent normalization processing.

[0039] After obtaining the brightness offset estimate, brightness normalization compensation processing needs to be performed on each road surface image frame. Pixel values ​​in brighter areas are compressed according to the offset, while those in darker areas are stretched accordingly, so that the brightness distribution of the entire frame falls within the expected range. The image frames after normalization processing are then arranged in the original acquisition order to form an image sequence with relatively stable brightness changes. Such a sequence better meets the brightness continuity requirements of subsequent optical flow calculations and also reduces unnecessary brightness spurious differences.

[0040] After brightness normalization, initial optical flow data can be constructed based on pixel changes between adjacent frames. Optical flow calculation itself only reflects the displacement of pixels in the image coordinate system, without distinguishing between displacement caused by object motion and displacement caused by camera attitude changes. Since drones experience continuous pitch, yaw, and roll changes during flight, directly using the initial optical flow data will result in a noticeable overall drift pattern in the optical flow vector field. This pattern cannot accurately reflect the local changes in road surface texture.

[0041] Therefore, attitude correction processing needs to be introduced after optical flow construction. The attitude parameters record the changes in yaw, pitch, and roll angles, from which attitude features describing attitude changes can be extracted, and a rotation matrix can be constructed to perform coordinate transformation on the optical flow. The attitude transformation matrix can be expressed as:

[0042] ;

[0043] in, , , These represent the yaw angle, pitch angle, and roll angle, respectively. , , These represent rotation matrices about the three coordinate axes, This is the attitude transformation matrix obtained from the combination of three-axis attitudes. Based on this matrix, the initial optical flow vector can be mapped from the camera's viewpoint to a unified reference coordinate system. The correction operation can be written as:

[0044] ;

[0045] in, The initial optical flow vector, This is the optical flow vector after attitude correction, used to reflect the displacement trend of the road surface texture in a unified coordinate system. Through this transformation, the overall drift in the optical flow related to attitude changes is weakened, and what is retained is mainly the true relative motion of the road surface texture in the sequence.

[0046] After completing the attitude transformation, the polarization correction optical flow data needs to be converged sequentially according to frame order to form a continuous optical flow vector field. This optical flow vector field has been processed in both brightness and attitude dimensions, resulting in a more stable inter-frame structure.

[0047] In another possible implementation, certain areas of asphalt and aggregate particles produce strong reflections, and traditional global brightness normalization struggles to handle such localized biases. Therefore, before brightness compensation, local block segmentation is performed on the image frame, and the brightness shift characteristics of each sub-block are calculated based on the exposure correlation of each sub-block. This block-based estimation results in a more refined brightness compensation structure, ensuring stable optical flow calculations in high-contrast regions.

[0048] In terms of attitude correction, this embodiment can further introduce dynamic weights based on attitude gradients. That is, the attitude transformation matrix is ​​weighted frame by frame, ensuring stricter alignment for frames with drastic attitude changes, while maintaining lightweight mapping for frames with gentle attitude changes. The purpose of this is not to increase computational complexity, but to reduce overcorrection under minor attitude variations, thus maintaining a consistent optical flow direction distribution. The resulting optical flow vector field is more suitable for subsequent local angle aggregation of crack orientation because noise and reflection interference in the optical flow are further suppressed. Through this finer-grained brightness processing and more flexible attitude mapping strategy, this embodiment is suitable for environments with complex road conditions and frequent changes in lighting.

[0049] Step S3: Calculate the texture stability index based on the optical flow vector field and perform texture stability region weight reduction processing to generate crack direction analysis optical flow data for crack direction extraction.

[0050] Specifically, based on the optical flow vectors corresponding to each pixel in the optical flow vector field, the direction and amplitude components of each optical flow vector are extracted, and statistical operations are performed on the direction and amplitude components in the neighborhood of each pixel to generate a texture stability index for characterizing local texture stability. The texture stability region to which the corresponding pixel belongs is determined according to the texture stability index, and the optical flow vectors in the optical flow vector field that are in the texture stability region are weighted according to the texture stability index to generate reduced-weight optical flow data for weakening the influence of the texture stability region. The reduced-weight optical flow data is converged according to the sequence frame order to form crack direction analysis optical flow data for crack direction extraction.

[0051] Furthermore, based on the texture stability index, the texture stability region to which the corresponding pixel belongs is determined, and the optical flow vectors in the optical flow vector field located in the texture stability region are weighted according to the texture stability index to generate reduced-weighted optical flow data for weakening the influence of the texture stability region. This includes: classifying the texture stability of each pixel in the optical flow vector field based on the texture stability index to determine the texture stability region to which each pixel belongs; mapping the optical flow vectors in the texture stability region to the texture stability index, and performing weight adjustment on the direction and amplitude components of the optical flow vectors according to the texture stability index to generate corresponding reduced-weighted optical flow vectors; and converging the reduced-weighted optical flow vectors according to the sequence frame order to form reduced-weighted optical flow data for crack orientation extraction.

[0052] In this embodiment of the invention, the texture of the road surface is not uniform; the particle structure, wear level, and surface material combination in different regions all affect the optical flow vector. Therefore, before proceeding to crack direction extraction, it is necessary to first screen these texture differences to ensure that subsequent processing is based on a more stable optical flow representation. The main task of this stage is to identify texture regions that maintain strong local consistency in the sequence and to appropriately reduce the weight of these regions so that crack-related directional changes can be more prominent.

[0053] In practice, the optical flow vector corresponding to the position of each pixel can be extracted from the optical flow vector field first. The optical flow vector typically consists of two parts: a direction component and an amplitude component. The direction component expresses the angular information of the pixel displacement, and the amplitude component expresses the magnitude of the displacement change. For road scenes, both components may be affected by high-frequency texture noise, so statistical calculations need to be performed within a smaller neighborhood. A common approach is to calculate the variance or circular variance of the direction component within a fixed neighborhood window, while simultaneously performing a local distribution evaluation on the amplitude component. This statistical result can reflect the trend of texture changes in the neighborhood; if the change is small, it indicates that the texture in that neighborhood tends to be stable.

[0054] To facilitate the quantification of this type of stability, a texture stability index can be constructed based on the statistical values ​​of the direction and amplitude components. Texture stability is not a single value, but a parameter describing the trend of local consistency, and its calculation method can be written as:

[0055] ;

[0056] in, Indicates pixel position Texture stability index This indicates the degree of local dispersion of the optical flow direction component within this neighborhood. This indicates the degree of local dispersion of the amplitude component. and The coefficient is used to balance the contributions of the two components. This structure provides a continuous set of stationarity indices for evaluating the texture representation of different regions of an image.

[0057] After obtaining the texture stability index, it is necessary to further determine which regions belong to the texture stability zone. These regions typically exhibit obvious consistency in the sequence and do not undergo drastic shifts with frame changes, thus having limited significance for crack direction extraction. When classifying based on index values, several intervals are generally set, and pixels whose stability index falls within a specific range are marked as texture stability zones. The marked areas often correspond to relatively uniform aggregate structures in asphalt or pavement sections with low wear. Although these areas have visually continuous texture, they do not carry explicit features indicating crack direction.

[0058] After identifying texture-stable regions, the optical flow vectors within these regions need to be weighted to reduce their impact on the overall directional distribution during subsequent crack direction extraction. In this step, the optical flow vectors in the texture-stable regions are mapped to corresponding texture stability indices, assigning lower weights to regions with higher indices. The weighting process can be described as scaling the optical flow direction and amplitude components separately, so that both components contribute less in subsequent steps. The adjusted optical flow vectors are called deweighted optical flow vectors.

[0059] When reconstructing the reduced-weighted optical flow vector, a frame-by-frame processing approach can be adopted. This involves individually reducing the weight of texture-stable regions in each frame, and then converging the processing results into a reduced-weighted optical flow data sequence in the original frame order. The purpose of this is to maintain the temporal continuity of the sequence, ensuring that the weighting process does not disrupt the structural regularity of optical flow changes over time. The reduced-weighted data does not replace the original optical flow vector; instead, it serves as input specifically for directional analysis, providing a cleaner directional field for subsequent crack orientation extraction.

[0060] The entire processing chain logically consists of three parts: extracting the optical flow direction and amplitude components, performing texture stability calculation and region partitioning, and performing weight reduction processing to generate weighted optical flow data. Among these three parts, texture stability evaluation is a core task because the direction of cracks is often manifested by sudden directional shifts or local gradient changes, while the optical flow direction changes in texture-stable regions are weak, making them prone to bias during the direction convergence stage. Therefore, weighting texture-stable regions can make the crack direction structure more prominent in the overall direction distribution.

[0061] The reduced-weighted optical flow data obtained after the above processing steps exhibits higher consistency between frames. Low-frequency texture motion caused by road surface noise in the orientation field is suppressed, while local orientation changes caused by cracks are enhanced in the sequence, allowing subsequent crack orientation analysis to be performed on a more stable data basis. In addition, the reduced-weighted optical flow data sequence still retains a complete temporal structure, thus enabling better integration into subsequent orientation clustering and morphological reconstruction processes.

[0062] Step S4: Based on the crack orientation analysis optical flow data, extract the crack orientation and introduce a sub-pixel optical flow differential operator to generate a crack depth variation index, forming crack morphology data.

[0063] Specifically, based on the optical flow vectors corresponding to each pixel in the crack orientation analysis optical flow data, optical flow direction components reflecting the local displacement direction distribution are extracted, and direction aggregation processing is performed on the optical flow direction components to generate crack orientation features that characterize the linear orientation of the crack. The crack orientation features are then processed in correspondence with the crack orientation analysis optical flow data, and a sub-pixel optical flow differential operator that characterizes the local differential gradient is introduced into the corresponding processing result. Differential calculation is performed on the optical flow vectors corresponding to each pixel to generate a crack depth change index that reflects the crack depth change. The crack orientation features and the crack depth change index are then converged according to the corresponding frame order to form crack morphology data for three-dimensional reconstruction of road cracks.

[0064] In this embodiment of the invention, the optical flow vectors do not exhibit uniform behavior in road scenes. Some areas have rough textures and random directions, while crack areas often show stable directional shifts. Therefore, it is necessary to perform centralized processing on the optical flow directions to highlight areas with consistent directions. The purpose of this step is not to directly generate crack morphology, but to lay the foundation for subsequent depth variation analysis.

[0065] In practice, the orientation component can be extracted from each pixel in the optical flow data for crack orientation analysis. The orientation component is angular information describing the directional trend of pixel displacement, while the amplitude component describes the magnitude of the displacement. Orientation components often exhibit strong coherence within local regions, especially near crack edges, thus requiring orientation aggregation. Orientation aggregation typically involves statistically analyzing the dominant directions of the orientation components within the neighborhood, thereby generating a local linear trend. As a concise aggregation operation, orientation aggregation can suppress highly random variations in optical flow direction, making crack-related directions more concentrated.

[0066] After directional aggregation, a crack orientation feature can be generated to characterize the linear orientation of the crack. This feature does not depend on a single pixel but is obtained by synthesizing multiple neighborhoods. The crack orientation feature contains both the dominant trend of the local orientation and a certain degree of continuity information, providing a directional reference for subsequent depth variation analysis. To make fuller use of the displacement structure in the optical flow, the crack orientation feature needs to be correlated with the aforementioned crack orientation analysis optical flow data, establishing a relationship between the orientation feature and the original pixel-level optical flow.

[0067] After the directional features are established, a sub-pixel optical flow differential operator can be introduced. This operator is used to extract the gradient changes of the optical flow vector at the sub-pixel scale. Crack depth often manifests as small gradient differences in the optical flow amplitude, and these changes are difficult to capture using ordinary-scale differencing operations. The sub-pixel optical flow differential operator is designed to compute subtle gradients of direction and amplitude within the pixel neighborhood, allowing depth changes to be reflected in the data. Simplified differential expressions can be used when needed, for example:

[0068] ;

[0069] in, pixel position The depth-related gradient value, This represents the spatial differential of the optical flow vector field at that location, including the directional gradient and the magnitude gradient. While not aiming for excessive precision, this formula effectively expresses the essential source of depth variations: the differential structure of the optical flow in the local region.

[0070] The sub-pixel optical flow differential operator is used to reconstruct the optical flow vector field at the sub-pixel level through interpolation, and to perform differential calculations on the optical flow direction and amplitude components within the pixel neighborhood to obtain the displacement gradient changes of the optical flow vector field. In conventional optical flow calculations, pixel displacement is usually represented with pixel-level precision. However, in real-world scenarios, many structural changes often manifest as very small displacement differences. Relying solely on pixel-level optical flow often fails to accurately reflect these details. The sub-pixel optical flow differential operator addresses this by performing a more refined sub-pixel-level reconstruction of the optical flow based on the existing optical flow vector field through interpolation, and then calculating the differential changes of the optical flow direction and amplitude components within the local neighborhood. This extracts the subtle displacement gradients that were originally hidden between pixels, making some imperceptible trends in the optical flow field clearer. In engineering applications, this type of operator is often used to characterize locally moving structures or minute deformations, such as identifying subtle boundary changes and analyzing the changing trends of structural textures. By introducing subpixel-scale differential computation, the ability to express subtle geometric changes can be improved without altering the overall optical flow structure.

[0071] After performing differential calculations, a crack depth variation index can be obtained to reflect changes in crack depth. This type of index typically exhibits a continuous variation along the crack direction, while fluctuating less in non-crack regions. By combining directional aggregation and gradient calculations, subtle changes in crack depth can be recovered relatively well.

[0072] Finally, the crack orientation features and crack depth variation indicators need to be converged frame by frame to form a unified crack morphology representation. This data structure typically includes orientation information, depth variation, and temporal sequence, which is directly valuable for subsequent 3D reconstruction of road cracks. Through convergence processing, the spatial and temporal structural representation of cracks becomes clearer, providing a stable foundation for spatial registration and ground coordinate alignment.

[0073] Overall, this stage of processing establishes a connection between directional features and depth gradients, enabling the accurate representation of crack geometry within the optical flow domain. This structured data will be used in the next stage for 3D registration, providing the foundational data for the final crack mapping.

[0074] Step S5: Perform ground coordinate registration based on the crack morphology data and generate road crack mapping results.

[0075] Specifically, based on the crack direction features and crack depth variation indicators corresponding to each frame in the crack morphology data, crack spatial features are extracted to characterize the spatial positional relationship between frames; the crack spatial features are correspondingly processed with attitude parameters used to characterize the flight attitude of the UAV, and ground coordinate registration relationships used to limit the spatial position of cracks are constructed according to the corresponding processing results to generate crack registration data under a unified coordinate system; the crack registration data is subjected to sequence aggregation processing according to the spatial continuity requirements to form road crack mapping results that reflect the direction and depth variation of road cracks.

[0076] In this embodiment of the invention, the preceding processing has already provided the crack orientation characteristics and crack depth variation indicators for each frame, but this information remains on the viewpoint-dependent image plane. To obtain mapping results that can be directly used for road assessment, it is necessary to introduce spatial positional relationships and construct a registration framework that expresses the crack geometry under a unified ground coordinate system. This step is equivalent to assigning a "spatial address" to the crack morphology.

[0077] In practical processing, crack spatial features describing the spatial relationship between frames can be extracted based on the crack orientation features and crack depth variation indices corresponding to each frame in the crack morphology data. Crack orientation features provide directional information, while depth variation indices provide local geometric undulations; combining these two allows inference of the relative positional changes of the crack between adjacent frames. During processing, a "node-segment" structure is typically constructed in a three-dimensional sense, connecting key points along the crack line in frame order and recording the orientation vector and relative depth of each key point. In this way, the crack spatial features include both the distribution along the crack line and the trend of change in the thickness direction.

[0078] Attitude information acts as a bridge in the registration process. While the attitude parameters recorded during UAV flight don't directly provide the crack location, they do reveal the geometric relationship between the camera's viewpoint and ground coordinates. A one-to-one mapping between the crack's spatial features and attitude parameters is needed, placing the crack features of each frame into the corresponding attitude state. In this way, the direction and depth changes of each crack can be correlated to the pitch angle, yaw angle, roll angle, and flight altitude at a given moment. Based on this mapping, a spatial mapping from the image plane to the ground plane can be constructed.

[0079] When expressing this mapping relationship, a homography matrix or rigid body transformation model is usually introduced. A common form of ground coordinate registration can be written as:

[0080] ;

[0081] in, , This represents the pixel coordinates of a point in the crack morphology data within the image coordinate plane. , This represents the planar position coordinates of the point in the ground coordinate system. The homography matrix corresponding to the ground coordinate registration relationship is determined by factors such as attitude parameters, camera intrinsic parameters, and flight altitude. This matrix form allows for the precise mapping of the crack's location in the image to the reference coordinate system of the road surface, laying the foundation for subsequent surveying and mapping representation.

[0082] After obtaining the crack registration results for a single frame, sequence aggregation along the time axis is required. A single image only shows the crack morphology of a cross-section, while road cracks often extend a long distance along the direction of traffic. During processing, the crack registration data can be stitched together frame by frame, merging and deduplicating the spatial point sets of the same crack in adjacent frames. A certain spatial neighborhood threshold is typically set to determine whether crack points in different frames belong to the same crack segment. After completing this process, a continuous crack spatial trajectory in a unified coordinate system can be obtained.

[0083] Depth information also needs to be preserved at this stage. The crack depth variation index, initially obtained from the optical flow differential structure, is now mapped to the ground coordinate system through registration, allowing a depth attribute to be attached to each spatial point of the crack. In this way, the crack morphology data is no longer a simple planar polyline, but a spatial curve with associated elevation. For road engineers, this representation is closer to the needs of on-site surveys and can be directly used to estimate crack penetration levels or predict disease development trends. After sequence aggregation is completed, the final road crack mapping results will be generated.

[0084] Preferably, the method further includes: performing crack segment aggregation processing on the road crack mapping results based on the crack direction and crack depth changes corresponding to each crack segment in the road crack mapping results, generating crack segment aggregation data to characterize the overall structure of the road cracks; extracting crack path features to describe the overall path continuity of the cracks based on the crack segment aggregation data, and constructing a crack path set to express the spatial distribution of cracks based on the crack path features; and performing corresponding processing on the crack path set and the road crack mapping results to generate structured output data of road cracks for subsequent road maintenance analysis.

[0085] In this embodiment of the invention, road crack mapping results often contain a large number of discrete crack segments. These crack segments are spatially close to each other, but are still regarded as independent units at the data level. If they are used directly for analysis, the interpretation cost will be relatively high. Therefore, it is necessary to reorganize the crack segments based on the mapping results and link the local segments with the overall structure. This part is accomplished by crack segment aggregation processing.

[0086] Crack segment aggregation processing first addresses the segment relationships within a single crack. Based on the crack orientation and depth variations of each segment, it can be determined whether adjacent crack segments belong to the same structural unit. A common approach is to introduce spatial proximity constraints and orientation consistency constraints, jointly assessing the distance between endpoints and the orientation angle. Depth variation information is also valuable here; when two crack segments are spatially close, have small orientation differences, and exhibit continuous depth variation trends, they can be merged into the same aggregation segment. Through this filtering process, previously scattered crack segments are combined into longer aggregated crack segment data, resulting in a clearer overall structural profile.

[0087] After obtaining the aggregated crack segment data, it is necessary to further extract crack path features that describe the overall path continuity. Crack path features no longer focus on the local attributes of individual segments, but rather on the extension morphology of the entire crack within the road area. Based on the aggregated crack segment results, one or more directed paths can be constructed, connecting the spatial order, directional changes, and depth changes of the aggregated segments into a continuous path. Path features typically include the path start and end coordinates, total path length, directional change sequence, and depth change curve. These elements, combined, can reflect the development trajectory of the crack on the road. Based on these features, a crack path set can be constructed to express the spatial distribution of cracks, organizing the cracks on the road into several clear path units.

[0088] The final step is to correlate the set of crack paths with the original road crack mapping results, generating structured output data that is easily accessible for subsequent maintenance decisions. In this stage, each crack path is associated with specific external attributes such as its road mileage location, lane, and adjacent structures, transforming the crack from a geometric object into a queryable and categorizable maintenance object. The structured output data can assign a unique identifier to each crack path, linking it to corresponding path features, depth indicators, and spatial location, enabling subsequent road maintenance analysis modules to directly perform grading evaluations or maintenance prioritization based on this data.

[0089] like Figure 2 As shown, this invention provides a road crack UAV detection and mapping system. The system includes: an acquisition unit, used to acquire a sequence of road surface images based on the UAV along a preset flight trajectory, and record the corresponding exposure parameters and attitude parameters to obtain a sequence of image data for brightness compensation; a compensation unit, used to perform brightness normalization compensation based on the sequence of image data and generate a brightness-compensated sequence of images, and construct an optical flow vector field in combination with the attitude parameters; a processing unit, used to calculate a texture stability index based on the optical flow vector field and perform texture stability region weight reduction processing to generate crack direction analysis optical flow data for crack direction extraction; an index generation unit, used to extract crack direction based on the crack direction analysis optical flow data and introduce a sub-pixel optical flow differential operator to generate a crack depth change index, forming crack morphology data; and a registration unit, used to perform ground coordinate registration based on the crack morphology data and generate road crack mapping results.

[0090] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 3As shown in the figure, the computer device includes a processor A01, a network interface A02, a display screen A04, an input device A05, a memory (not shown), and a database (not shown) connected via a system bus. The processor A01 provides computing and control capabilities. The network interface A02 interacts with external terminals or servers via a bidirectional communication link (indicated by double-headed arrows in the figure), receiving external data and sending processing results. The display screen A04 displays data processing results or system operating status information, and the input device A05 receives user-input commands or parameter information. The computer device's memory includes internal memory A03 and a non-volatile storage medium A06. The non-volatile storage medium A06 stores an operating system B01, a computer program B02, and a database (not shown). The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 stored in the non-volatile storage medium A06. The network interface A02 communicates with external terminals via a network connection. When the computer program B02 is executed by the processor A01, it implements a method for detecting and mapping road cracks using a drone.

[0091] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a microcontroller, chip, or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0092] The optional embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details described above. Within the scope of the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the protection scope of the embodiments of the present invention. It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the embodiments of the present invention will not further describe the various possible combinations.

[0093] Furthermore, various different embodiments of the present invention can be combined in any way, as long as they do not violate the spirit of the embodiments of the present invention, they should also be regarded as the content disclosed by the embodiments of the present invention.

Claims

1. A method for detecting and mapping road cracks using unmanned aerial vehicles (UAVs), characterized in that, The method includes: The UAV collects a sequence of road surface images along a preset flight path and records the corresponding exposure and attitude parameters to obtain a sequence of image data for brightness compensation. Brightness normalization compensation is performed based on the sequence image data to generate a brightness-compensated sequence image, and an optical flow vector field is constructed by combining the attitude parameters. Based on the optical flow vector field, the texture stability index is calculated and the texture stability region is weighted down to generate crack orientation analysis optical flow data for crack orientation extraction. Based on the crack orientation analysis, the crack orientation is extracted from the optical flow data, and a sub-pixel optical flow differential operator is introduced to generate a crack depth variation index, forming crack morphology data. The sub-pixel optical flow differential operator is used to perform sub-pixel reconstruction of the optical flow vector field by interpolation, and to perform differential calculations on the optical flow direction component and amplitude component in the pixel neighborhood to obtain the displacement gradient change of the optical flow vector field. Based on the crack morphology data, ground coordinate registration is performed to generate road crack mapping results.

2. The road crack UAV detection and mapping method according to claim 1, characterized in that, Based on the UAV acquiring a sequence of road surface images along a preset flight path and recording the corresponding exposure and attitude parameters, a sequence of image data for brightness compensation is obtained, including: During the flight of the UAV along the preset flight path, continuous road surface image frames are acquired according to the preset image acquisition time sequence, and the exposure parameters and attitude parameters at the corresponding frame time are recorded synchronously when acquiring each road surface image frame. Based on the acquisition order, time tag pairing processing is performed on the road image frames, the exposure parameters, and the attitude parameters to generate sequence image tags for defining the continuity between frames; The road surface image frames with the sequence image tags, the corresponding exposure parameters, and the corresponding attitude parameters are aggregated to form a sequence image data for brightness compensation.

3. The road crack UAV detection and mapping method according to claim 1, characterized in that, Based on the sequence image data, brightness normalization compensation is performed to generate a brightness-compensated sequence image. An optical flow vector field is constructed using the attitude parameters, including: A brightness shift estimation process is performed on each road surface image frame in the sequence image data, and a brightness shift feature is generated to represent local brightness changes based on the corresponding exposure parameters; Based on the brightness offset feature, perform brightness normalization compensation processing on the corresponding road surface image frame to generate a brightness compensation image frame sequence corresponding to the original acquisition order. Based on the brightness-compensated image frame sequence, initial optical flow data for representing pixel displacement is constructed in the order of adjacent frames, and the corresponding attitude parameters are introduced into the initial optical flow data to perform attitude correction processing, thereby generating the corresponding optical flow vector field.

4. The road crack UAV detection and mapping method according to claim 3, characterized in that, The corresponding attitude parameters are introduced into the initial optical flow data to perform attitude correction processing, generating the corresponding optical flow vector field, including: Based on the attitude parameters, attitude features are extracted to characterize the attitude changes of the UAV, and an attitude transformation matrix is ​​constructed to reflect the viewpoint differences between adjacent frames based on the attitude features. The attitude transformation matrix is ​​applied to the initial optical flow data, and coordinate system alignment processing is performed on the optical flow vectors corresponding to each pixel in the initial optical flow data to generate correction optical flow data that reflects the displacement of road surface texture. The optical flow data that corrects polarization is aggregated according to the sequence of frames to form the optical flow vector field.

5. The road crack detection and mapping method using an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, Based on the optical flow vector field, a texture stability index is calculated and a weighting process is performed on the texture stability region to generate optical flow data for crack orientation extraction, including: Based on the optical flow vector corresponding to each pixel in the optical flow vector field, the direction component and amplitude component of each optical flow vector are extracted, and statistical operations are performed on the direction component and amplitude component in the neighborhood of each pixel to generate a texture stability index for characterizing local texture stability. The texture stability index determines the texture stability region to which the corresponding pixel belongs, and performs weight adjustment processing on the optical flow vector in the optical flow vector field that is in the texture stability region according to the texture stability index, generating deweighted optical flow data to reduce the influence of the texture stability region. The reduced-weight optical flow data is aggregated according to the sequence frame order to form crack orientation analysis optical flow data for crack orientation extraction.

6. The road crack UAV detection and mapping method according to claim 5, characterized in that, Based on the texture stability index, the texture stability region to which the corresponding pixel belongs is determined, and the optical flow vectors in the optical flow vector field located in the texture stability region are weighted according to the texture stability index to generate deweighted optical flow data to reduce the influence of the texture stability region, including: Based on the texture stability index, the texture stability of each pixel in the optical flow vector field is classified to determine the texture stability region to which each pixel belongs. The optical flow vector in the texture stable region is mapped to the texture stability index, and the direction and amplitude components of the optical flow vector are weighted according to the texture stability index to generate the corresponding weighted optical flow vector. The reduced-weight optical flow vectors are converged according to the sequence frame to form reduced-weight optical flow data for crack orientation extraction.

7. The road crack detection and mapping method using unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, Based on the crack orientation analysis optical flow data, the crack orientation is extracted and a sub-pixel optical flow differential operator is introduced to generate a crack depth variation index, forming crack morphology data, including: Based on the optical flow vectors corresponding to each pixel in the crack orientation analysis optical flow data, optical flow direction components that reflect the local displacement direction distribution are extracted, and direction aggregation processing is performed on the optical flow direction components to generate crack orientation features that characterize the linear orientation of the crack. The crack orientation features are correlated with the crack orientation analysis optical flow data, and a sub-pixel optical flow differential operator for characterizing local differential gradients is introduced into the corresponding processing results. Differential calculations are performed on the optical flow vectors corresponding to each pixel to generate a crack depth change index that reflects the crack depth change. The crack orientation features and crack depth variation indicators are aggregated according to the corresponding frame order to form crack morphology data for three-dimensional reconstruction of road cracks.

8. The road crack detection and mapping method using unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, Based on the crack morphology data, ground coordinate registration is performed to generate road crack mapping results, including: Based on the crack orientation features and crack depth variation index corresponding to each frame in the crack morphology data, crack spatial features are extracted to characterize the spatial positional relationship between frames. The spatial features of the cracks are mapped to attitude parameters used to characterize the flight attitude of the UAV, and ground coordinate registration relationships for defining the spatial location of the cracks are constructed based on the mapping results, generating crack registration data in a unified coordinate system. The crack registration data is subjected to sequence aggregation processing according to spatial continuity requirements to form road crack mapping results that reflect the changes in road crack direction and crack depth.

9. The road crack detection and mapping method using unmanned aerial vehicles according to claim 8, characterized in that, The method further includes: Based on the crack direction and crack depth changes of each crack segment in the road crack mapping results, crack segment aggregation processing is performed on the road crack mapping results to generate crack segment aggregation data for characterizing the overall structure of road cracks. Based on the aggregated data of the crack segments, crack path features are extracted to describe the overall path continuity of the cracks, and a set of crack paths is constructed to express the spatial distribution of the cracks based on the crack path features. The set of crack paths is matched with the road crack mapping results to generate structured output data of road cracks for subsequent road maintenance analysis.

10. A road crack unmanned aerial vehicle (UAV) detection and mapping system, characterized in that, The system includes: The acquisition unit is used to acquire a sequence of road surface images based on the UAV along a preset flight path, and record the corresponding exposure parameters and attitude parameters to obtain a sequence of image data for brightness compensation. The compensation unit is used to perform brightness normalization compensation based on the sequence image data and generate brightness-compensated sequence images, and to construct an optical flow vector field in combination with the attitude parameters; The processing unit is used to calculate the texture stability index based on the optical flow vector field and perform texture stability region weight reduction processing to generate crack direction analysis optical flow data for crack direction extraction. The index generation unit is used to extract the crack orientation based on the crack orientation analysis optical flow data and introduce a sub-pixel optical flow differential operator to generate a crack depth variation index, thus forming crack morphology data; wherein... The subpixel optical flow differential operator is used to reconstruct the optical flow vector field in a subpixel manner by interpolation based on the optical flow vector field, and to perform differential calculations on the optical flow direction component and amplitude component in the pixel neighborhood to obtain the displacement gradient change of the optical flow vector field. The registration unit is used to perform ground coordinate registration based on the crack morphology data and generate road crack mapping results.

Images