A road disease recognition and positioning method based on cross-scale feature enhancement

Abstract

The application relates to the technical field of road disease detection, and discloses a road disease identification and positioning method based on cross-scale feature enhancement, which comprises the following steps: acquiring background features of a to-be-processed image and disease features representing disease details of a crack area; when it is determined that a suspected crack area belongs to an ultra-wide crack area according to the physical width of the suspected crack area and the local response state of the background features in the suspected crack area, decoupling the features of the background features in the ultra-wide crack area into stable background features and abnormal texture features; synthesizing virtual background features in the ultra-wide crack area according to the stable background features, so as to fill the background loss in the ultra-wide crack area; and performing constrained cross-scale fusion on the disease features and the structural features of the suspected crack area, generating continuous disease enhanced features, and outputting a positioning result covering the ultra-wide crack area, so that the problem of background loss caused by the insufficient large-scale receptive field is effectively solved, and the continuity of the disease features is enhanced.

Description

Technical Field

[0001] This application relates to the field of road defect detection technology, and more specifically, to a road defect identification and localization method based on cross-scale feature enhancement. Background Technology

[0002] In the field of road defect identification and localization, existing technologies typically employ vehicle-mounted cameras to acquire road surface images, which are then input into pre-trained cross-scale feature enhancement networks. These networks extract features at different scales and establish spatial dependencies between large-scale background and small-scale details to achieve defect boundary regression. However, in actual highway inspection operations, transverse or longitudinal ultra-wide cracks exceeding 20 cm in width are frequently encountered. When faced with such ultra-wide cracks, existing technologies struggle because the crack width exceeds the background range corresponding to the receptive field of the large-scale feature map. Each activation unit in the large-scale feature map cannot treat the crack as a whole as background, nor can it fully consider it as a detail, resulting in background loss when establishing spatial dependencies. Abnormal textures within the crack interfere with background representation, causing the enhanced features to lack cross-scale continuity constraints within the crack's interior region, leading to the crack being broken down into multiple fragmented feature clusters. Ultimately, this results in multiple discontinuous small-sized detection boxes being output within the crack, misidentifying a wide crack as several independent defects, or causing boundary regression divergence due to the lack of background constraints at the crack center, leading to a severe underestimation of crack length and width. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this application provides a road defect identification and localization method based on cross-scale feature enhancement, which can solve the technical problems in existing technologies where background loss and feature fragmentation caused by ultra-wide cracks lead to crack localization failure and size underestimation.

[0004] Firstly, this application provides a method for identifying and locating road defects based on cross-scale feature enhancement, including: Obtain background features and disease features that characterize the details of the crack area in the image to be processed; Identify suspected crack regions in the image to be processed. When the suspected crack region is determined to be an ultra-wide crack region based on the physical width of the suspected crack region and the local response state of the background features within the suspected crack region, the background features within the ultra-wide crack region are decoupled into stable background features and abnormal texture features. Based on stable background features, virtual background features are synthesized within the ultra-wide crack region to fill in the background gaps within the ultra-wide crack region; Using virtual background features as a unified background reference, a constrained cross-scale fusion is performed with disease features and structural features of suspected crack areas to generate continuous disease enhancement features. Boundary regression is performed based on the continuous enhancement characteristics of the disease to output the localization results covering the ultra-wide crack area.

[0005] Optionally, the step of determining whether a suspected crack region belongs to an ultra-wide crack region based on the physical width of the suspected crack region and the local response state of the background features within the suspected crack region includes: Obtain the physical width of the suspected crack region mapped onto the image to be processed; At each spatial location of the background feature, the local response volatility within its neighborhood is calculated to generate a response volatility state map. Compare the physical width with the physical coverage width of the background features on the image to be processed; If the physical width exceeds a preset percentage threshold of the physical coverage width, and the average response volatility of the suspected crack area in the response volatility state diagram exceeds a first preset threshold, then the suspected crack area is determined to be an ultra-wide crack area.

[0006] Optionally, the step of decoupling background features within the ultra-wide crack region into stable background features and anomalous texture features includes: Identify the mask region covering the ultra-wide crack region and its adjacent areas on both sides from the background features; Based on the edge and center features of the masked area, the reference features of the normal road surface background and the reference features of the abnormal texture are determined. Calculate the similarity between the features of each spatial location within the mask area and the reference features of the normal road surface background and the reference features of the abnormal texture; Based on similarity, the spatial location within the mask area is divided into a stable background area or a disease internal texture area, and the background features are further divided into stable background features and abnormal texture features based on the division results.

[0007] Optionally, the step of synthesizing virtual background features within the ultra-wide crack region based on stable background features to fill in the background gaps within the ultra-wide crack region includes: Obtain information on the distribution of stable background features on both sides of the ultra-wide crack region; Based on distribution information, multiple virtual interpolation lines are established on both sides of the ultra-wide crack region; For the missing background location within the ultra-wide crack region, determine the two intersection points between the virtual interpolation line passing through the missing background location and the stable background feature; Based on the spatial distance from the missing background location to the two intersection points, the stable background features at the two intersection points are weighted and synthesized to generate virtual background features.

[0008] Optionally, the step of synthesizing virtual background features within the ultra-wide crack region based on stable background features to fill in the background gaps within the ultra-wide crack region further includes: Each sub-region within the suspected crack area is marked as a strongly reconstructed region, a transitional region, or a normal region. State propagation is performed along the extension direction of the suspected crack region, based on the topological connectivity between the various sub-regions. If the transition region is directly or indirectly connected to the strong reconstruction region, then the transition region is upgraded to a reconstruction region. If the transition region is not connected to any of the strongly reconstructed regions, then the transition region will be downgraded to a regular region. Based on the labeling results after state propagation, the stable background features are weighted to generate virtual background features that include transitional background features.

[0009] Optionally, the step of marking each sub-region within the suspected crack area as a strongly reconstructed region, a transition region, or a normal region includes: Sub-regions with a physical width greater than a second preset threshold are marked as strongly reconstructed regions; Sub-regions whose physical widths of suspected crack areas are between the third preset threshold and the second preset threshold are marked as transition regions, where the third preset threshold is less than the second preset threshold. Sub-regions with a physical width less than a third preset threshold are marked as normal regions.

[0010] Optionally, the step of weighting stable background features based on the labeling results after state propagation to generate virtual background features including transitional background features includes: After state propagation, at the physical boundary marked as directly adjacent to the reconstructed region and the normal region, a preset length is extended to both sides along the extension direction of the suspected crack region to form a feature smooth buffer zone. The smoothing weight coefficients are determined based on the spatial location within the feature smoothing buffer band and the distance to the strongly reconstructed region. Based on the smoothing weight coefficient, the stable background features are weighted to generate virtual background features that include transitional background features.

[0011] Optionally, the step of using virtual background features as a unified background reference and performing constrained cross-scale fusion with disease features and structural features of suspected crack areas to generate continuous disease enhancement features includes: Using the local structural information of the suspected crack area as a constraint, the virtual background features are fused with the structural features of the suspected crack area, and the structural features are mapped onto the virtual background features during the fusion process. The disease features are then fused with the mapped virtual background features again. During the fusion process, a continuity constraint is applied along the extension direction of the ultra-wide crack region, so that the edge details in the disease features form a continuous feature response along the extension direction, generating continuous disease enhancement features.

[0012] Optionally, the step of applying continuity constraints along the extension direction of the ultra-wide crack region during the fusion process includes: The main orientation of the ultra-wide crack region was extracted based on the structural characteristics of the suspected crack region. Strengthen the structural features of suspected crack areas along a direction parallel to the main trunk, so that multiple local responses in the disease features are connected into a continuous feature band along the main trunk.

[0013] Optionally, the step of performing boundary regression based on continuous disease enhancement characteristics to output localization results covering the ultra-wide crack region includes: Using the continuous feature bands formed along the extension direction of the ultra-wide crack region in the continuous disease enhancement features as the positioning basis, the minimum bounding rectangle surrounding the ultra-wide crack region is predicted. Non-maximum suppression is performed on all predicted minimum bounding rectangles, and the minimum bounding rectangle covering the ultra-wide crack region is retained as the localization result.

[0014] This application proposes a feature decoupling and virtual background synthesis scheme to address the problem of missing background in ultra-wide cracks by introducing a cross-scale feature enhancement mechanism. By decoupling background features into a stable background and anomalous textures, and synthesizing a virtual background using background information from both sides, a unified background reference is successfully constructed. Through cross-scale fusion of structural and continuity constraints, fragmented disease features are integrated into continuous feature bands, thereby achieving complete localization of ultra-wide cracks. This effectively solves the problems of crack localization fragmentation and size underestimation caused by limited receptive field in existing technologies, improving the accuracy of road defect identification. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating a road defect identification and localization method based on cross-scale feature enhancement, provided as an embodiment of this application. Detailed Implementation

[0016] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0017] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0018] In the field of road defect identification and localization, vehicle-mounted cameras are typically used to acquire road surface images in scenarios such as highways, and these images are processed using a cross-scale feature enhancement network. This network extracts features at different scale levels; for example, small scales correspond to crack details, while large scales correspond to the global road surface background, thus establishing a spatial dependency relationship between the large-scale background and small-scale details. However, in actual highway inspection operations, wide transverse or longitudinal ultra-wide cracks are frequently encountered, formed after severe road surface subsidence or repair. The exposed base layer or repair material within these cracks exhibits a strong response different from that of normal road surfaces. Because the crack width exceeds the background range corresponding to the receptive field of the large-scale feature map, the large-scale feature map cannot treat the crack as a whole as background. Furthermore, the abnormal texture within the crack causes a strong local response, resulting in a missing background when establishing spatial dependencies. This missing background does not mean that image pixels are absent, but rather that in the large-scale background representation upon which cross-scale fusion relies, the normal road surface reference, which should exist continuously, is interrupted by the abnormal texture within the ultra-wide crack, making this area no longer stably interpretable as the road surface background. This results in the enhanced features lacking cross-scale continuity constraints within the crack region, causing the crack to be broken down into multiple fragmented feature clusters, ultimately leading to the localization of the fracture and underestimation of its size.

[0019] See Figure 1 This application discloses a road defect identification and localization method based on cross-scale feature enhancement, including: Obtain background features and disease features that characterize the details of the crack area in the image to be processed; Identify suspected crack regions in the image to be processed. When the suspected crack region is determined to be an ultra-wide crack region based on the physical width of the suspected crack region and the local response state of the background features within the suspected crack region, the background features within the ultra-wide crack region are decoupled into stable background features and abnormal texture features. Based on stable background features, virtual background features are synthesized within the ultra-wide crack region to fill in the background gaps within the ultra-wide crack region; Using virtual background features as a unified background reference, a constrained cross-scale fusion is performed with disease features and structural features of suspected crack areas to generate continuous disease enhancement features. Boundary regression is performed based on continuous disease enhancement characteristics to output localization results covering ultra-wide crack areas.

[0020] Cross-scale feature enhancement refers to fusing features extracted from images at different resolutions to simultaneously utilize macroscopic context and microscopic details. Taking road crack detection as an example, small-scale features are more likely to preserve local details such as crack edges, crack tips, small bifurcations, and damaged particle boundaries, while large-scale features are more likely to preserve the overall texture continuity of the lane area, pavement direction, shadow distribution trends, and the overall background relationship around the defect. Background features typically refer to features that characterize the smooth texture of a normal road surface at a large scale; essentially, they represent a stable representation by the network that the area resembles a complete road surface rather than a defect. Defect features, on the other hand, refer to detailed features such as crack edges and damaged contours captured at a small scale, emphasizing defect attributes such as local abrupt changes, boundary discontinuities, and texture fragmentation. Ultra-wide crack regions refer to specific defect regions whose physical width exceeds the coverage of the existing network's large-scale receptive field and are accompanied by complex texture interference.

[0021] The physical width exceeding the receptive field coverage can be understood as follows: when the network observes the region at a large-scale layer, it can no longer continuously see the normal road surface on one side of the crack from the other side. Therefore, it can no longer rely on the complete background to constrain the feature representation inside the crack. For example, although ordinary fine cracks are visible in the image, the background on both sides can still maintain continuity in large-scale features, so cross-scale fusion can usually still form a complete disease representation; however, ultra-wide cracks will interrupt this continuous background, thus causing incomplete localization problems in subsequent steps.

[0022] The images to be processed can be acquired by vehicle-mounted cameras installed at the front, top, or inside the windshield of the inspection vehicle. During acquisition, the cameras continuously capture images of the road surface area ahead, and the video stream is decoded frame by frame into the images to be processed. Background features and disease features can be obtained from the outputs of different layers of the same cross-scale feature enhancement network, or they can be extracted jointly by the backbone network and the feature pyramid structure. The outputs of deeper layers serve as the source of background features, while the outputs of shallower or intermediate layers serve as the source of disease features.

[0023] The suspected crack area can be determined by existing disease candidate detection branches, semantic segmentation results, edge aggregation results, or connected component analysis results, as long as the area that may correspond to the main body of the crack can be delineated on the image. The so-called local response state refers to whether the background features exhibit obvious unstable, abrupt, or mixed response distribution within the region, reflecting whether the region has lost the smoothness and consistency that a normal background should have.

[0024] The synthetic virtual background features are not generated out of thin air and are unrelated to the real image. Instead, based on the still reliable and stable background features around the ultra-wide crack region, the missing background reference inside the crack is continuously filled in within the feature space, so that the region can regain a background base that can be used for cross-scale fusion.

[0025] Among them, structural features can represent the extension direction, main axis shape, boundary orientation, regional connectivity, or skeleton distribution of suspected crack areas, which are used to constrain the continuous propagation of disease details along the real crack structure, rather than spreading to unrelated areas during the fusion process.

[0026] Boundary regression can be performed by the detection head, and the output can be a rectangular box, a rotated box, or a region boundary that matches the contour of the disease, as long as it can completely cover the ultra-wide crack area.

[0027] The operational logic of the above overall technical solution is as follows: First, features at different scales are extracted from the image to be processed, and suspected crack areas are evaluated upon discovery. Once it is confirmed that the area belongs to an ultra-wide crack region that would cause background loss, proactive intervention is initiated to decompose the chaotic background features within the area, stripping away abnormal texture features while retaining stable background features. The chaos here mainly manifests in the fact that different locations within the same ultra-wide crack may correspond to exposed base layers, repair materials, accumulated dust, shadows, or edge fragmentation areas. These elements will produce significantly different responses in the deep feature map, causing the network to be unable to uniformly interpret them as background. Subsequently, a complete virtual background is reconstructed using the retained stable background features to fill the original background gaps. This virtual background preferably extends inward along the texture trends, brightness variations, and spatial continuity of the normal road surface on both sides of the ultra-wide crack, thereby allowing the crack interior to regain a background reference consistent with the surrounding road surface. Finally, under this complete and unified virtual background reference, the details and structural features of the defect are integrated to form a continuous feature representation, thereby guiding the detection head to accurately output a bounding box surrounding the entire defect. Because a continuous background constraint already exists during fusion, the internal details of cracks that were originally easily disassembled will be reorganized into a whole, thus significantly reducing the probability of location fractures and size underestimation.

[0028] Specifically, when synthesizing virtual background features, the normal background features surrounding the ultra-wide crack region can be used to fill it inwards using a bilinear interpolation algorithm. Subsequently, this virtual background feature is stitched and fused with the defect features along the channel dimension, with convolution operations used to smooth the boundaries, generating continuous defect enhancement features. The mask range corresponding to the ultra-wide crack region can be determined on the background feature map first, and one or more rings of stable background features can be extracted from the outside of this mask as the filling source. The role of bilinear interpolation is to ensure that the area to be filled inside the crack obtains smoothly transitioning feature values ​​from the surrounding stable background based on spatial proximity, thus avoiding abrupt changes caused by directly copying edge features. If the ultra-wide crack has obvious lateral or longitudinal extension, background extension can be prioritized along the lateral or longitudinal direction corresponding to the crack width, making the virtual background more consistent with the continuous trend of road paving texture. After filling, the virtual background feature and defect features are stitched together along the channels, and then convolution operations are used to smooth the stitched boundary area, reducing the potential seam between the virtual background and real defect details.

[0029] The above technical solution effectively overcomes the feature fragmentation problem caused by missing background, enabling the final boundary regression to output a location result that completely surrounds the ultra-wide crack based on continuous enhanced features, thus avoiding location fragmentation and size underestimation. After introducing a virtual background, the details of the lesion no longer need to rely on isolated local anomaly responses to maintain their existence, but can form a continuous expression around the same background reference. Therefore, the location result will converge from multiple fragmented candidate regions into a complete lesion region.

[0030] Optionally, the step of determining whether a suspected crack region belongs to an ultra-wide crack region based on the local response state of the suspected crack region and background features within the suspected crack region includes: Obtain the physical width of the suspected crack region mapped onto the image to be processed; At each spatial location of the background feature, the local response volatility within its neighborhood is calculated to generate a response volatility state map. Compare the physical width with the physical coverage width of the background features on the image to be processed; If the physical width exceeds a preset percentage threshold of the physical coverage width, and the average response volatility of the suspected crack area in the response volatility state diagram exceeds a first preset threshold, then the suspected crack area is determined to be an ultra-wide crack area.

[0031] Specifically, the intrinsic and extrinsic parameters of the vehicle-mounted camera, as well as its installation attitude relative to the road surface, are first acquired. Then, based on the assumption that the road surface is approximately a local plane, the boundary points of the suspected crack area in the image are back-projected onto the road surface coordinate system. For two boundary points located in the same transverse direction, their actual distance on the road surface can be calculated, and this distance is used as the physical width of the suspected crack area. If the inspection system also has a vehicle attitude sensor or a high-precision positioning module, the projection results can be corrected using synchronized attitude data to reduce the impact of vehicle bumps and pitch changes on the width estimation.

[0032] The neighborhood range can be understood as a local window selected on the background feature map centered on the current spatial location, used to observe whether the features around that location are stable and consistent. If the feature vectors at different locations within this window are close to each other, it indicates that the location is more likely to belong to the normal background; if the differences are significant, it indicates that the location is more likely to be affected by abnormal texture interference. The response fluctuation state map can be a single-channel map corresponding to the spatial size of the background feature map, used to record the degree of fluctuation at each location.

[0033] The physical coverage width refers to the lateral range that the background feature layer currently used to provide background reference can stably cover on the original road surface. This coverage width can be obtained through a combination of network structure analysis and camera spatial mapping. That is, first determine the receptive range of a single spatial location on the feature map of this layer in the original image, and then map this receptive range onto the actual road surface to obtain its corresponding physical coverage capability.

[0034] Both the preset ratio threshold and the first preset threshold can be obtained through offline sample statistics. Specifically, the relative relationship between the physical width of the crack and the physical coverage width of the background feature, as well as the average response fluctuation level of the crack's internal region, can be statistically analyzed from labeled ordinary crack samples and ultra-wide crack samples. Then, a threshold that can effectively distinguish between the two types of samples can be selected as the judgment criterion. If the statistical results differ for different road types and different camera installation methods, threshold configuration tables for corresponding scenarios can be established and called according to the scenario during deployment.

[0035] The above technical solution elevates the judgment criteria from a single image pixel level to a dimension combining physical space and deep feature response. This not only eliminates scale errors caused by changes in shooting perspective, but also accurately identifies complex disease areas that truly exceed the network's receptive field and whose internal texture is sufficient to destroy the background representation, thereby significantly improving the accuracy and adaptability of ultra-wide crack judgment.

[0036] Furthermore, the step of decoupling background features within the ultra-wide crack region into stable background features and anomalous texture features includes: Identify the mask region covering the ultra-wide crack region and its adjacent areas on both sides from the background features; Based on the edge and center features of the masked area, the reference features of the normal road surface background and the reference features of the abnormal texture are determined. Calculate the similarity between the features of each spatial location within the mask area and the reference features of the normal road surface background and the reference features of the abnormal texture; Based on similarity, the spatial location within the mask area is divided into a stable background area or a disease internal texture area, and the background features are further divided into stable background features and abnormal texture features based on the division results.

[0037] The masked area is preferably slightly larger than the ultra-wide crack area itself. This is to cover not only the main defect but also the background areas on both sides of the defect that still retain normal pavement properties, facilitating the extraction of reference features from the same local context. The masked area can be obtained by expanding the projection range of the suspected crack area onto the background feature map. The expansion direction preferably covers both the crack width direction and the extension direction to ensure sufficient edge reference information. The masked area can be obtained by expanding the projection boundary of the suspected crack area onto the background feature map outwards. The expansion scale can be adaptively determined based on the size of the suspected crack area; that is, the wider the crack, the larger the corresponding outer reference area, but excessive expansion should be avoided to prevent the introduction of other irrelevant structures. Next, the feature vectors of the outermost edge of the masked area are extracted and averaged as reference features for the normal pavement background; simultaneously, the feature vectors of the center position of the masked area are extracted and averaged as reference features for the abnormal texture. To improve the representativeness of the reference features, the edges can be further filtered to remove locations with obvious abnormal response fluctuations, avoiding the inclusion of shadow edges, marker edges, or other occasional interferences into the normal background reference; the center location can also be preferentially selected to be the area inside the ultra-wide crack body and far from the boundary transition zone, so as to reduce the contamination of abnormal texture reference by edge blending features.

[0038] Edge features are typically closer to the normal road surface surrounding the defect, while central region features are typically closer to the exposed base layer, repair material, or broken texture inside the ultra-wide crack. Therefore, they are suitable as reference benchmarks for stable backgrounds and abnormal textures, respectively. To improve the purity of the reference features, obvious shadows, lane markings, or other non-road surface interference areas can be avoided when extracting edge features, and locations within the main body of the defect and far from the boundary transition zone should be prioritized when extracting central region features.

[0039] Here, similarity can be achieved using inner product, cosine similarity, or other methods that reflect the degree of feature proximity. In practice, to avoid a single outlier affecting the overall judgment, the reference features can first be averaged or robustly aggregated, and then the proximity of each position to the two types of reference features can be compared.

[0040] The segmentation results can form a spatial mask image. Locations identified as stable background areas are retained in the stable background feature branch, while locations identified as internal texture areas of disease are retained in the abnormal texture feature branch. For transitional locations between the two reference types, a buffer zone processing strategy can be implemented, such as assigning them lower weights to participate in stable background reconstruction, to reduce the impact of boundary misclassification. Based on the spatial segmentation results, the original background features are precisely split into pure stable background features and independent abnormal texture features using a spatial attention mechanism or mask multiplication. If a spatial attention mechanism is used, stable background areas can be assigned higher retention weights, while internal texture areas of disease can be assigned lower retention weights, resulting in stable background features more suitable for virtual background synthesis. Simultaneously, assigning higher retention weights to internal texture areas of disease yields independent abnormal texture feature branches, which can be used for subsequent interference suppression or to assist in disease structure analysis.

[0041] The above technical solution dynamically generates a reference baseline by utilizing the local contextual information of the affected area, avoiding reliance on absolute thresholds. This spatial partitioning method based on relative similarity can effectively resist interference from changes in external lighting and differences in road surface materials, ensuring that abnormal textures in the interfering background can be completely removed under various complex working conditions.

[0042] Furthermore, the step of synthesizing virtual background features within the ultra-wide crack region based on stable background features to fill in the background gaps within the ultra-wide crack region includes: Obtain information on the distribution of stable background features on both sides of the ultra-wide crack region; Based on distribution information, multiple virtual interpolation lines are established on both sides of the ultra-wide crack region; For the missing background location within the ultra-wide crack region, determine the two intersection points between the virtual interpolation line passing through the missing background location and the stable background feature; Based on the spatial distance from the missing background location to the two intersection points, the stable background features at the two intersection points are weighted and synthesized to generate virtual background features.

[0043] The distribution information mainly refers to the spatial geometric orientation and feature response distribution of the normal road surface background on both sides of the ultra-wide crack area. This distribution information can be obtained by segmenting the suspected crack area in the image to be processed and extracting the feature sampling bands near the crack boundaries from the stable background feature map. Specifically, highway pavement images can be acquired using vehicle-mounted industrial cameras, inspection vehicle roof array cameras, or roadside fixed camera equipment, followed by prior defect candidate region detection to obtain the suspected crack area and its boundaries. Subsequently, a background sampling band of a preset width is set on each side of the crack boundary, and the main texture direction, brightness variation trend, feature response strength, and local continuity are statistically analyzed within the sampling band to form the distribution information.

[0044] The virtual interpolation line is used to establish a spatial mapping relationship between the two stable backgrounds within the missing area. Its direction is typically perpendicular to the local extension direction of the crack to ensure the interpolation process can cross the missing area with the shortest possible distance. "Perpendicular to the local extension direction of the crack" does not require absolute geometric perpendicularity, but rather means establishing a connection along the shortest possible width across the crack, so that the two ends of the same line correspond to the two reference points most likely belonging to the same continuous background. The intersection point is the location where the virtual interpolation line intersects with the feature boundaries of the two stable backgrounds, representing the effective reference source used for interpolation. Spatial distance measures the proximity of the missing location to the two reference sources; the closer the distance, the closer the background attributes at that location should be to the corresponding reference source. For example, with a wide, curved crack, if a fixed horizontal direction is used to fill the bend, it may incorrectly connect the two non-corresponding backgrounds. However, using a virtual interpolation line that follows the local direction can more accurately correspond the originally continuous road surface texture on both sides of the crack, thereby reducing synthesis errors.

[0045] In one specific embodiment, the skeleton lines of the ultra-wide crack region can be extracted first, and their normal directions can be calculated at regular pixel intervals along the skeleton lines. The skeleton lines can be obtained by thinning a binary mask of the suspected crack region, or by extracting the central axis of the region; their function is to characterize the extension path of the crack's main trunk. If the crack region has branches, the normal directions can be independently established for each branch skeleton line, and the main direction is selected preferentially based on local continuity in the intersection region. These normals are extended to both sides until they intersect with the effective boundaries of stable background features, thereby establishing multiple virtual interpolation lines.

[0046] The effective boundary refers to the boundary location outside the crack area where the stable background feature response is continuous and not disturbed by other disease areas. If shadows, oil stains, or other disease areas are encountered during the extension of the normal, the intersection point in that direction can be marked as a low-confidence reference point, and if necessary, the reference point corresponding to the adjacent normal can be used to supplement it. For any missing background location within the ultra-wide crack area, find the nearest one or more virtual interpolation lines and determine the intersection points of these lines on both sides of the boundary. If the missing location is exactly between two adjacent virtual interpolation lines, the nearest line can be used first, or the results of two adjacent lines can be combined for smoothing to avoid strip-like artifacts caused by local sparse sampling. Subsequently, the Euclidean distance from the missing location to the intersection points on both sides is calculated.

[0047] Assuming the distance to the left intersection point is D1 and the distance to the right intersection point is D2, the weight of the feature at the left intersection point can be set as D2 / (D1+D2), and the weight of the feature at the right intersection point can be set as D1 / (D1+D2). This means that the closer the missing location is to the left intersection point, the closer the generated virtual background feature will be to the stable left background feature; the closer it is to the right intersection point, the closer it will be to the stable right background feature. By linearly weighting the stable background feature vectors at both intersection points using the calculated weights, the virtual background feature at the missing location can be obtained. For ease of implementation, the stable background feature vector can be understood as a multi-channel representation of that location on the feature map, such as a combination of texture response, brightness response, and local structure response. For example, if a missing background location is closer to the left boundary, it will inherit more of the left road surface texture and brightness features during synthesis, while retaining a certain proportion of the right-side features, creating a natural transition in the central area without a noticeable left-right splicing boundary.

[0048] Furthermore, the step of synthesizing virtual background features within the ultra-wide crack region based on stable background features to fill in the background gaps within the ultra-wide crack region also includes: Each sub-region within the suspected crack area is marked as a strongly reconstructed region, a transitional region, or a normal region. State propagation is performed along the extension direction of the suspected crack region, based on the topological connectivity between the sub-regions. If the transition region is directly or indirectly connected to the strong reconstruction region, then the transition region is upgraded to a reconstruction region. If the transition region is not connected to any of the strongly reconstructed regions, then the transition region will be downgraded to a regular region. Based on the labeling results after state propagation, the stable background features are weighted to generate virtual background features that include transitional background features.

[0049] Strong reconstruction regions refer to local areas with extremely large widths, severe background loss, and requiring complete background reconstruction. Their typical characteristic is that the lateral span of the crack significantly exceeds the continuously perceptible range of stable background features, making it difficult for the backgrounds on both sides to directly correspond on the feature map. Regular regions refer to local areas with narrow widths, still continuous background features, and no need for reconstruction. These regions typically retain relatively complete original textures and brightness transitions. Transitional regions are those in between, with uncertain states that depend on the surrounding context to determine the final processing method. Topological connectivity reflects the spatial connectivity of each sub-region. State propagation utilizes this connectivity to transmit definite states along the crack's extension direction to ambiguous regions. First, a complete mask of the suspected crack area can be obtained from the road image segmentation results. Then, combined with the skeleton line or main trunk fitting results, the crack can be divided into several sub-regions according to a fixed length or inflection point position. If the crack has branches, the branch points can be retained as special nodes in the topological map, and the connection relationships between each branch and the main trunk can be recorded.

[0050] Specifically, the suspected crack area can be divided into multiple sub-regions of equal length along its main direction. The length of the sub-region can be selected as L based on the image resolution, the average crack width, and the accuracy requirements of subsequent processing. L can be set to a range that can simultaneously take into account local details and overall stability. If the image resolution is high, a shorter sub-region length can be used to improve local management accuracy; if the image resolution is low, the sub-region length can be appropriately increased to avoid too few samples in a single sub-region. By evaluating the attributes of each sub-region, an initial labeling state is assigned to it. The attributes can at least include the average physical width, maximum physical width, background continuity, the width variation with adjacent sub-regions, and whether there is a significant interruption in stable background features.

[0051] The physical width can be calculated by combining pixel width with camera calibration parameters and shooting distance. If the calibration information of the inspection vehicle has been integrated, the image measurement value can be directly mapped to the actual road surface size. Subsequently, an undirected graph is constructed with sub-regions as nodes and adjacency relationships as edges. A state propagation algorithm, such as breadth-first search, is executed on the graph. Starting from all nodes marked as strong reconstruction regions, the graph is traversed outwards along the edges. If a transition region node is encountered, its state is updated to a reconstruction region, and propagation continues outwards; if a regular region node is encountered, propagation in that direction stops. After propagation of all strong reconstruction regions is complete, the remaining transition region nodes in the graph are checked. Since these nodes have not received any strong reconstruction signals, they are in relatively isolated, narrow crack segments, so their states are uniformly modified to regular regions. Finally, based on the updated state graph, background compositing is performed only on the parts marked as strong reconstruction regions and reconstruction regions, while the original background features are directly retained in the regular regions.

[0052] Furthermore, the step of marking each sub-region within the suspected crack area as a strongly reconstructed region, a transitional region, or a normal region includes: Sub-regions with a physical width greater than a second preset threshold are marked as strongly reconstructed regions; Sub-regions whose physical widths of suspected crack areas are between the third preset threshold and the second preset threshold are marked as transition regions, where the third preset threshold is less than the second preset threshold. Sub-regions with a physical width less than a third preset threshold are marked as normal regions.

[0053] The second preset threshold represents the critical width at which background features completely fail. When the crack width exceeds this value, the large-scale receptive field can no longer cross the crack to obtain any effective background information from the opposite side. The third preset threshold represents the initial width at which background features begin to be significantly disturbed. When the crack width is less than this value, the receptive field can easily cover both sides of the crack, and the background features remain highly continuous. The width range between the two corresponds to a semi-failed, semi-continuous, and ambiguous state of background features. Both the second and third preset thresholds can be determined jointly through sample statistics and network response analysis. Specifically, sample images of highway cracks with different widths, materials, and lighting conditions can be collected first, and the true width range of each sample can be obtained by combining manual annotation. Then, the background continuity performance of the cross-scale feature enhancement network on cracks of different widths can be observed, such as whether the similarity of features on both sides of the crack decreases significantly, whether there are background response holes in the central area, and whether there is abnormal enhancement at the disease boundary. Based on these analysis results, a width limit that makes background features basically fail can be determined as the second preset threshold; at the same time, a smaller width limit that begins to significantly affect background continuity can be determined as the third preset threshold. To improve stability, candidate thresholds can be counted separately for different road surface types, and then a set of unified thresholds suitable for the target application scenario can be selected, or a threshold configuration table can be created according to road type for use.

[0054] Specifically, by using two width thresholds with clear physical meaning, continuous crack width variations are mapped to three discrete state levels. This hard thresholding provides a clear and reliable initial input for subsequent topology state propagation. In highway defect detection applications, different types of cracks exhibit significant differences in width, enabling the rapid and accurate conversion of these complex width variations into network-understandable state labels.

[0055] Furthermore, the step of weighting the stable background features based on the labeling results after state propagation to generate virtual background features including transitional background features includes: After state propagation, at the physical boundary marked as directly adjacent to the reconstructed region and the normal region, a preset length is extended to both sides along the extension direction of the suspected crack region to form a feature smooth buffer zone. The smoothing weight coefficients are determined based on the spatial location within the feature smoothing buffer band and the distance to the strongly reconstructed region. Based on the smoothing weight coefficient, the stable background features are weighted to generate virtual background features that include transitional background features.

[0056] The physical boundary refers to the spatial boundary where the state changes abruptly. It can be directly determined by the sub-region label boundary after state propagation, or further refined to pixel-level boundaries or feature map grid boundaries to improve transition zone positioning accuracy. The feature smoothing buffer band is an artificially defined area spanning this boundary, designed to provide sufficient spatial span for feature gradation. The preset length determines the smoothness of the gradation process; a longer length results in a more natural transition, but may blur some real details. Therefore, this preset length can be obtained through offline verification, such as testing various buffer band lengths in a sample set, comparing the boundary artifact suppression effect and detail preservation effect, and selecting a length that balances both as the system configuration value. If the road image resolution or acquisition height varies significantly, it can also be uniformly set according to the physical length and then converted to pixel length. The smoothing weight coefficient is a dynamically changing value with spatial location, used to control the mixing ratio of the synthetic background and the original background within the buffer band. The trend of change should meet the following requirements: when approaching a strongly reconstructed region, more emphasis should be placed on using synthesized virtual background features; when approaching a normal region, more emphasis should be placed on using original real background features; and when in the middle, the two should gradually transition in a certain proportion. For example, if a reconstructed region is adjacent to a normal region, and the boundary is located at a certain position in the direction of crack extension, then within a limited length range before and after the boundary, the binary treatment of full reconstruction or full preservation should no longer be used, but rather gradually mixed to form a smooth transition.

[0057] In the specific implementation process, after determining the physical boundary between the reconstructed region and the regular region, a predetermined number of pixels can be extended along the direction of the crack into both the reconstructed region and the regular region, forming a feature smoothing buffer band. This predetermined number can be pre-set as a length B based on the image resolution, the average crack width, and the sensitivity of the boundary regression network to the seam. If the image resolution is high, this length can be appropriately increased to ensure a sufficiently smooth transition process; if the reconstructed region itself is short, the length of the buffer band can be trimmed to avoid the buffer band covering the entire sub-region. Within this buffer band, the distance along the line from each pixel position to the nearest strong reconstructed region core is calculated.

[0058] The core of the strongly reconstructed region can be understood as the location within the strongly reconstructed region that best represents the center of the missing background, such as the region at the center of the ultra-wide segment along the skeleton line. To achieve a smooth transition, linear interpolation or a non-linear sigmoid function can be used to generate smoothing weight coefficients. If linear interpolation is used, the weights change uniformly along the length of the buffer band, which is simple to implement and easy to control; if a non-linear sigmoid function is used, a softer transition can be formed near the boundary, which is suitable for handling cases with particularly sensitive boundaries. Assuming linear interpolation is used, the smoothing weight coefficient is set to 1 at the end of the buffer band near the strongly reconstructed region, that is, the synthesized virtual background features are used entirely; at the end of the buffer band near the regular region, the smoothing weight coefficient is set to 0, that is, the original real background features are used entirely. For the middle position within the buffer band, the weight coefficient gradually changes between 0 and 1. Subsequently, using this smoothing weight coefficient, the synthesized virtual background features and the original background features are weighted and fused to generate the final transition background features, which are used as the final background representation of the buffer band region.

[0059] To improve implementation stability, local continuity checks can be performed on the generated transitional background features. For example, check whether the texture direction change at the boundaries before and after the buffer band is smooth and whether there are abnormal peaks in the brightness response. If a significant abrupt change is found at a certain boundary, the buffer band length can be appropriately extended or a gentler weight change method can be used. For instance, at the location where an ultra-wide crack core segment connects to a narrow crack segment, after the above buffer band processing, the features will gradually transition from complete reconstruction to complete realism preservation. The boundary regression network sees a continuously changing background, rather than a suddenly switched spliced ​​interface.

[0060] Furthermore, the steps of using virtual background features as a unified background reference and performing constrained cross-scale fusion with disease features and structural features of suspected crack areas to generate continuous disease enhancement features include: Using the local structural information of the suspected crack area as a constraint, the virtual background features are fused with the structural features of the suspected crack area, and the structural features are mapped onto the virtual background features during the fusion process. The disease features are then fused with the mapped virtual background features again. During the fusion process, a continuity constraint is applied along the extension direction of the ultra-wide crack region, so that the edge details in the disease features form a continuous feature response along the extension direction, generating continuous disease enhancement features.

[0061] In this context, local structural information refers to the geometric attributes of the suspected crack region in image space, such as its morphological distribution, boundary contours, and skeleton topology. Structural features are the network's characteristic representation of these geometric attributes. In the initial stage of cross-scale fusion, structural features are first mapped onto virtual background features. Specifically, spatial attention mechanisms or feature concatenation combined with convolutional dimensionality reduction can be used to superimpose structural features representing crack morphology onto the corresponding spatial locations of virtual background features. This process imbues the originally smooth virtual background with prior location and morphological constraints related to the defects, causing background features to generate specific response biases in the potential crack region. Subsequently, small-scale defect features containing rich details are introduced and fused again with the mapped virtual background features. Since the background features now possess structural constraints, the defect features will not blindly spread to unrelated normal road surface areas during fusion, but will be confined to the range defined by the structural features. Simultaneously, during the re-fusion process, continuity constraints are applied along the extension direction of the ultra-wide crack region. This continuity constraint can be achieved by designing receptive fields in specific directions or introducing direction-aware graph convolution, aiming to guide the network to find feature correlations along the depth direction of the crack.

[0062] Through the above technical solutions, cross-scale fusion is no longer a simple addition or splicing of feature maps, but a directional enhancement process guided by structural priors. Virtual background features provide a unified reference benchmark, eliminating interference from abnormal textures; structural features provide spatial range constraints, preventing feature divergence; and continuity constraints ensure the coherence of disease details in the extension direction.

[0063] Furthermore, the step of applying continuity constraints along the extension direction of the ultra-wide crack region during the fusion process includes: The main orientation of the ultra-wide crack region was extracted based on the structural characteristics of the suspected crack region. Strengthen the structural features of suspected crack areas along a direction parallel to the main trunk, so that multiple local responses in the disease features are connected into a continuous feature band along the main trunk.

[0064] The main orientation refers to the primary extension direction of ultra-wide cracks on a macroscopic scale. In actual highway pavements, cracks may extend laterally, longitudinally, or obliquely, and may have local bends. To extract this main orientation, principal component analysis can be performed on the structural feature map of suspected crack areas, or a skeleton extraction algorithm can be used to obtain the central axis of the crack, and then the tangent direction of each segment of the central axis can be calculated as the local main orientation. After obtaining the main orientation, the structural features need to be enhanced along this direction. In practice, directional asymmetric convolution kernels (such as 1×N or N×1 strip convolution kernels) or strip pooling operations can be used. The directions of these operations are dynamically aligned to the extracted main orientation, thereby aggregating contextual information along the crack extension direction on the feature map. When the defect features are fused with the directionally enhanced structural features, those originally isolated, locally broken, or occluded high-response points will transmit information to each other under the guidance of the main orientation and thus be connected.

[0065] Furthermore, the steps for performing boundary regression based on continuous disease enhancement characteristics to output localization results covering the ultra-wide crack region include: Using the continuous feature bands formed along the extension direction of the ultra-wide crack region in the continuous disease enhancement features as the positioning basis, the minimum bounding rectangle surrounding the ultra-wide crack region is predicted. Non-maximum suppression is performed on all predicted minimum bounding rectangles, and the minimum bounding rectangle covering the ultra-wide crack region is retained as the localization result.

[0066] In practical implementation, continuous disease enhancement features can be input into the detection head. When performing anchor-box matching or predicting center points without anchor boxes, the detection head can extract global boundary information within a complete feature response area. Based on these continuous feature bands, the network regresses the minimum bounding rectangle that can enclose the entire ultra-wide crack region. This minimum bounding rectangle can be a horizontal rectangle or a rotated rectangle with a rotation angle to more closely fit the tilted crack. During prediction, the network typically generates multiple candidate rectangles at different locations on the feature map. To eliminate redundancy, non-maximum suppression (NMS) is applied to these candidate rectangles. Specifically, the candidate rectangles are sorted according to their confidence scores, and the rectangle with the highest confidence score is selected as the benchmark. The intersection-union ratio (IU) of the other rectangles with the benchmark is calculated. If the IU is greater than a set threshold, these rectangles are considered to predict the same target and are discarded; the remaining rectangles are the optimal minimum bounding rectangles that can completely cover the ultra-wide crack region, serving as the final localization result.

[0067] Through the above technical solution, the boundary regression process makes full use of the global view brought by the continuous feature band. The detection head can treat the entire ultra-wide crack as a unified whole for boundary regression. The predicted minimum bounding rectangle can completely and accurately surround the entire disease area, which significantly improves the accuracy and reliability of automated road disease detection and provides real and reliable disease size data for subsequent road maintenance decisions.

[0068] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for identifying and locating road defects based on cross-scale feature enhancement, characterized in that, include: Obtain background features and disease features that characterize the details of the crack area in the image to be processed; Identify suspected crack regions in the image to be processed. When the suspected crack region is determined to be an ultra-wide crack region based on the physical width of the suspected crack region and the local response state of the background features in the suspected crack region, the background features in the ultra-wide crack region are decoupled into stable background features and abnormal texture features. Based on the stable background features, virtual background features are synthesized within the ultra-wide crack region to fill the background gaps within the ultra-wide crack region; The virtual background features are used as a unified background reference and are fused with the disease features and the structural features of the suspected crack area in a constrained cross-scale manner to generate continuous disease enhancement features. Boundary regression is performed based on the continuous disease enhancement characteristics to output the positioning results covering the ultra-wide crack area.

2. The road defect identification and localization method based on cross-scale feature enhancement according to claim 1, characterized in that, The step of determining that the suspected crack region belongs to an ultra-wide crack region based on the physical width of the suspected crack region and the local response state of the background features within the suspected crack region includes: Obtain the physical width of the suspected crack region mapped onto the image to be processed; At each spatial location of the background feature, the local response volatility within its neighborhood is calculated to generate a response volatility state diagram. The physical width is compared with the physical coverage width of the background feature on the image to be processed; If the physical width exceeds a preset percentage threshold of the physical coverage width, and the average response volatility of the suspected crack region in the response volatility state diagram exceeds a first preset threshold, then the suspected crack region is determined to belong to the ultra-wide crack region.

3. The road defect identification and localization method based on cross-scale feature enhancement according to claim 1, characterized in that, The step of decoupling the background features into stable background features and anomalous texture features within the ultra-wide crack region includes: A mask region covering the ultra-wide crack region and its adjacent regions on both sides is determined from the background features; Based on the edge features and center region features of the masked area, reference features for the normal road surface background and reference features for the abnormal texture are determined. Calculate the similarity between the features of each spatial location within the mask area and the reference features of the normal road surface background and the reference features of the abnormal texture; Based on the similarity, the spatial location within the mask area is divided into a stable background area or a disease internal texture area, and the background features are further divided into stable background features and abnormal texture features based on the division results.

4. The road defect identification and localization method based on cross-scale feature enhancement according to claim 1, characterized in that, The step of synthesizing virtual background features within the ultra-wide crack region based on the stable background features to fill in the background gaps within the ultra-wide crack region includes: Obtain the distribution information of the stable background features on both sides of the ultra-wide crack region; Based on the distribution information, multiple virtual interpolation lines are established on both sides of the ultra-wide crack region; For the missing background location within the ultra-wide crack region, determine two intersection points between the virtual interpolation line passing through the missing background location and the stable background feature; Based on the spatial distance from the missing background location to the two intersection points, the stable background features at the two intersection points are weighted and synthesized to generate the virtual background features.

5. The road defect identification and localization method based on cross-scale feature enhancement according to claim 1, characterized in that, The step of synthesizing virtual background features within the ultra-wide crack region based on the stable background features to fill in the background gaps within the ultra-wide crack region further includes: Each sub-region within the suspected crack area is marked as a strongly reconstructed region, a transitional region, or a normal region; State propagation is performed along the extension direction of the suspected crack region, based on the topological connectivity between the various sub-regions. If the transition region is directly or indirectly connected to the strong reconstruction region, then the transition region is upgraded to a reconstruction region; If the transition region is not connected to any of the strongly reconstructed regions, then the transition region is downgraded to a regular region. Based on the labeling results after state propagation, the stable background features are weighted to generate the virtual background features that include transitional background features.

6. The road defect identification and localization method based on cross-scale feature enhancement according to claim 5, characterized in that, The step of marking each sub-region within the suspected crack area as a strongly reconstructed region, a transition region, or a normal region includes: The sub-regions whose physical width is greater than the second preset threshold are marked as the strong reconstruction regions; The sub-region whose physical width of the suspected crack region is between the third preset threshold and the second preset threshold is marked as the transition region, wherein the third preset threshold is less than the second preset threshold; Sub-regions whose physical width is less than the third preset threshold are marked as normal regions.

7. The road defect identification and localization method based on cross-scale feature enhancement according to claim 6, characterized in that, The step of weighting the stable background features based on the labeling results after state propagation to generate the virtual background features including transitional background features includes: After state propagation, at the physical boundary marked as directly adjacent to the reconstructed region and the normal region, a predetermined length is extended to both sides along the extension direction of the suspected crack region to form a characteristic smooth buffer zone. The smoothing weight coefficient is determined based on the spatial location within the feature smoothing buffer band and the distance to the strongly reconstructed region; The stable background features are weighted according to the smoothing weight coefficient to generate the virtual background features that include transitional background features.

8. The road defect identification and localization method based on cross-scale feature enhancement according to claim 1, characterized in that, The step of using the virtual background features as a unified background reference and performing constrained cross-scale fusion with the disease features and the structural features of the suspected crack area to generate continuous disease enhancement features includes: Using the local structural information of the suspected crack region as a constraint, the virtual background features are fused with the structural features of the suspected crack region, and the structural features are mapped onto the virtual background features during the fusion process; The disease features are then fused with the mapped virtual background features again. During the fusion process, a continuity constraint is applied along the extension direction of the ultra-wide crack region, so that the edge details in the disease features form a continuous feature response along the extension direction, thereby generating the continuous disease enhancement features.

9. A road defect identification and localization method based on cross-scale feature enhancement according to claim 8, characterized in that, The step of applying continuity constraints along the extension direction of the ultra-wide crack region during the fusion process includes: Based on the structural characteristics of the suspected crack region, the main orientation of the ultra-wide crack region is extracted; The structural features of the suspected crack area are enhanced along a direction parallel to the main trunk, so that multiple local responses in the disease features are connected into a continuous feature band along the main trunk.

10. The road defect identification and localization method based on cross-scale feature enhancement according to claim 1, characterized in that, The step of performing boundary regression based on the continuous disease enhancement features and outputting the localization result covering the ultra-wide crack region includes: Using the continuous feature bands formed along the extension direction of the ultra-wide crack region in the continuous disease enhancement features as the positioning basis, predict the minimum bounding rectangle surrounding the ultra-wide crack region. Non-maximum suppression is performed on all predicted minimum bounding rectangles, and the minimum bounding rectangle covering the ultra-wide crack region is retained as the localization result.

Images