Road surface disease image generation method and system based on statistical distribution characteristic constraint and multi-strategy fusion

Abstract

This application relates to the field of image processing technology, and more particularly to a method and system for generating road surface distress images based on statistical distribution feature constraints and multi-strategy fusion. The method balances sample data of different road surface distress types by augmenting sample data of scarce categories; it determines the spatial positional relationship of selected target distress sample categories in the sample data based on the statistical distribution characteristics of the selected target distress sample categories in the sample data, ensuring that the multiple objects generated in the synthesized image follow the coexistence probability and spatial distance distribution rules of the real samples; and it determines the corresponding target fusion strategy based on the complexity of the target generation region, enabling the fusion algorithm to match an appropriate fusion method according to the complexity of the target object region itself, thereby avoiding splicing marks and inconsistencies in lighting texture caused by a single fusion strategy. The aim is to solve the problem of how to generate high-quality road surface distress images with reasonable spatial layout and smooth fusion boundaries.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and system for generating road surface distress images based on statistical distribution feature constraints and multi-strategy fusion. Background Technology

[0002] With the rapid development of transportation infrastructure construction, pavement defect detection has become a key technology for ensuring road safety. Traditional detection methods rely on manual inspection, which suffers from low efficiency and strong subjectivity. In recent years, deep learning-based detection algorithms have made significant progress, but their performance is heavily dependent on large-scale labeled datasets. Currently, publicly available labeled datasets show significant class imbalances in pavement defect samples and lack defect samples under complex scenarios (such as changes in lighting, severe weather, and pavement background interference), thus limiting the performance of detection algorithms.

[0003] In related technical solutions, Generative Adversarial Networks (GANs) are applied to pavement defect identification, making it possible to generate defect sample data. Currently, the StyleGAN series of models has attracted attention due to its superior performance in generating high-quality, high-resolution images.

[0004] However, in practical applications, the inventors found that directly using the collected road surface damage images as input to StyleGAN introduces a large amount of background information that interferes with the learning of damage features, resulting in blurry damage details that are difficult to control precisely. At the same time, high-resolution damage images significantly reduce the computational efficiency of the model, which restricts the practicality and effectiveness of data augmentation.

[0005] Therefore, this application proposes a new method for generating pavement distress images, aiming to generate high-quality pavement distress images with reasonable spatial layout and soft fusion boundaries. Summary of the Invention

[0006] The main objective of this application is to provide a method for generating pavement distress images based on statistical distribution feature constraints and multi-strategy fusion, aiming to solve the problem of how to generate high-quality pavement distress images with reasonable spatial layout and soft fusion boundaries.

[0007] To achieve the above objectives, this application provides a method for generating pavement distress images based on statistical distribution feature constraints and multi-strategy fusion, the method comprising:

[0008] Data augmentation is performed on target disease samples belonging to scarce categories in the road surface disease sample dataset. The scarce category of target disease samples is divided based on the proportion of each disease category in the road surface disease sample dataset.

[0009] Based on the statistical distribution characteristics of pavement distress categories in the amplified pavement distress sample dataset, the spatial positional relationship of multiple target distress sample categories in the pavement background image is determined. The statistical distribution characteristics include the coexistence probability and spatial distance distribution of the target distress sample categories, and the spatial positional relationship is determined based on the coexistence probability and the spatial distance distribution.

[0010] Determine the complexity of the target generation region for the target disease sample category, and determine the corresponding target fusion strategy based on the complexity;

[0011] Based on the spatial relationship, the target fusion strategy is used to fuse the target disease sample categories into the road surface background image to generate a road surface disease image.

[0012] Optionally, the spatial distance distribution includes the Euclidean distance between the center points of the target objects, the degree of overlap between the target generation regions, and the relative angle between the target generation regions. The steps for determining the spatial positional relationship include:

[0013] The combination of target disease categories to be synthesized is determined based on the coexistence probability.

[0014] Based on the Euclidean distance, the degree of overlap, and the relative angle, the relative positional relationship of each target disease category in the target disease category combination in the road surface background image is determined.

[0015] Optionally, determining the complexity of the target generation region for the target disease sample category, and determining the corresponding target fusion strategy based on the complexity of the target generation region, includes:

[0016] Obtain the edge density and texture variance of the target generation region for the target disease sample category;

[0017] The complexity is obtained by averaging the sum of the edge density and the texture variance and then normalizing it.

[0018] The target fusion strategy is determined based on the preset range in which the complexity falls.

[0019] Optionally, the preset interval includes a high complexity interval, a medium complexity interval, and a low complexity interval, and determining the target fusion strategy based on the preset interval in which the complexity falls includes:

[0020] When the complexity falls within the high complexity range, the selected target fusion strategy is multi-scale Poisson fusion or texture-aware fusion.

[0021] When the complexity is in the medium complexity range and the area of ​​the target generation region is greater than the preset area threshold, the selected target fusion strategy is multi-scale Poisson fusion.

[0022] When the complexity falls within the low complexity range, the selected target fusion strategy is either illumination-consistent fusion or progressive Alpha fusion.

[0023] Optionally, the multi-scale Poisson fusion includes the following steps:

[0024] The target generation region and the road background image are decomposed into a multi-scale pyramid; the gradient field of the target generation region at each scale in the multi-scale pyramid is calculated; based on the gradient field and the pixel value of the background region on the fusion boundary, the Poisson equation of the corresponding scale is solved; and the solution results of each scale are used for pyramid reconstruction and fusion based on the Poisson equation.

[0025] The texture-aware fusion includes the following steps:

[0026] Determine the target generation region of the target disease sample category and the texture features of the road surface background image; adjust the texture attributes of the target generation region according to the texture features to match the road surface background image;

[0027] The illumination uniformity fusion includes the following steps:

[0028] In the LAB color space, the brightness channel of the target generation region is adjusted to match the lighting conditions of the road background image. The color of the target generation region is matched with the color distribution of the road background image. A multi-layer fusion strategy is used to fuse the adjusted target generation region into the road background image.

[0029] The progressive Alpha fusion includes the following steps:

[0030] A directed distance field is constructed for the target generation region. Based on this distance field, a nonlinear transparency function is called to generate a progressive mask. The progressive mask is then used to progressively fuse the target generation region and the road background image.

[0031] A directed distance field is constructed for the target generation region. Based on this distance field, a nonlinear transparency function is called to generate a progressive mask. The progressive mask is then used to progressively fuse the target generation region and the road background image.

[0032] Optionally, the data type of the pavement defect sample dataset includes scarce categories and sufficient categories, and the steps for dividing the scarce categories and sufficient categories include:

[0033] Determine the ratio of the number of samples for each type of pavement distress to the total number of samples in the pavement distress sample dataset;

[0034] Pavement distress sample types with ratios less than a preset threshold are identified as scarce categories.

[0035] Otherwise, it is determined to be the sufficient category, wherein the pavement defect sample data corresponding to the sufficient category is not expanded.

[0036] Optionally, the step of data augmentation for target defect samples belonging to various scarce categories in the pavement defect sample dataset includes:

[0037] An initial generated region consistent with the scarce category is generated by a generative adversarial network, which employs an adaptive data augmentation strategy to prevent the discriminator from overfitting.

[0038] The initial generated region is subjected to quality screening to remove samples that are blurry, distorted, or have abnormal features;

[0039] The filtered samples are merged with the original samples belonging to the scarce category to form an amplified pavement disease sample dataset.

[0040] Optionally, the generated road surface defect image may include at least one environmental effect, including different lighting conditions, weather conditions, and road surface stains.

[0041] Furthermore, to achieve the above objectives, this application also provides a computer system for implementing the road surface distress image generation method based on statistical distribution feature constraints and multi-strategy fusion as described above, characterized in that the computer system comprises:

[0042] The small sample disease category data augmentation module is used to augment the data of various target disease samples belonging to scarce categories in the road surface disease sample dataset. The target disease samples of scarce categories are divided based on the proportion of each disease category in the road surface disease sample dataset.

[0043] A multi-disease category combination and layout module is used to determine the spatial positional relationship of multiple target disease sample categories in a road background image based on the statistical distribution characteristics of road disease categories in the amplified road disease sample dataset. The statistical distribution characteristics include the coexistence probability and spatial distance distribution of the target disease sample categories, and the spatial positional relationship is determined based on the coexistence probability and the spatial distance distribution.

[0044] The fusion strategy selection module is used to determine the complexity of the target generation region of the target disease sample category, and determine the corresponding target fusion strategy based on the complexity.

[0045] The fusion module is used to fuse the target disease sample category into the road surface background image according to the spatial position relationship and the target fusion strategy to generate a road surface disease image.

[0046] This application has at least the following beneficial effects:

[0047] 1. Balance the sample data of different pavement distress types by augmenting the sample data of scarce categories;

[0048] 2. Based on the statistical distribution characteristics of the selected target disease sample categories in the sample data, determine the spatial positional relationship in the roadside background image, so that the multiple objects generated in the synthetic image follow the coexistence probability and spatial distance distribution law in the real sample;

[0049] 3. Based on the complexity of the target generation region, the corresponding target fusion strategy is determined, so that the fusion algorithm can match the appropriate fusion method according to the complexity of the target object region itself, thereby avoiding the splicing traces and inconsistencies in lighting texture caused by a single fusion strategy. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating the method for generating pavement distress images based on statistical distribution feature constraints and multi-strategy fusion, as described in an embodiment of this application.

[0051] Figure 2 This is an example image of pavement distress generated by the pavement distress image generation method based on statistical distribution feature constraints and multi-strategy fusion according to the embodiments of this application;

[0052] Figure 3 This is a schematic diagram of the computer system architecture for the road surface distress image generation method based on statistical distribution feature constraints and multi-strategy fusion proposed in this application.

[0053] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0054] To better understand the above technical solutions, exemplary embodiments of this disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of this disclosure to those skilled in the art.

[0055] First Embodiment

[0056] Reference Figure 1 This embodiment provides a method for generating road surface distress images based on statistical distribution feature constraints and multi-strategy fusion. The method includes the following steps:

[0057] S10, augment the target disease samples belonging to the scarce category in the road surface disease sample dataset, wherein the target disease samples of the scarce category are divided based on the proportion of each disease category in the road surface disease sample dataset.

[0058] In this embodiment, when acquiring images of road surface defect sample datasets, it is necessary to ensure uniform illumination, appropriate shooting angle, and image resolution that meets recognition requirements.

[0059] In this embodiment, the pavement defect sample dataset includes scarce categories and sufficient categories. The steps for dividing the scarce and sufficient categories include:

[0060] S11, determine the ratio of the number of samples of each type of pavement distress to the total number of samples in the pavement distress sample dataset;

[0061] S12, identify the road surface defect sample types with a ratio less than a preset threshold as scarce categories;

[0062] S13, otherwise, it is determined to be the sufficient category, wherein the pavement defect sample data corresponding to the sufficient category is not expanded.

[0063] In some alternative implementations, a ROI extractor is designed to extract diseased areas from the pavement disease sample dataset. First, the XML file of the dataset is read to obtain the bounding box and category information for each disease. Then, the ROI region containing the context is calculated based on the disease bounding box and resized as a square. Next, a binary mask is created for each disease within the ROI region. The extracted ROI image and mask are saved to the corresponding category directory. Following these steps, the entire dataset is traversed to extract all diseased area slices that meet the criteria, and the scarce and sufficient categories within the diseased area slices are selected.

[0064] In this embodiment, the data augmentation step includes:

[0065] S14, Generative Adversarial Network (GAN) generates initial disease samples consistent with the scarce category, wherein the GAN employs an adaptive data augmentation strategy to prevent the discriminator from overfitting.

[0066] S15, perform quality screening on the initial disease samples to remove samples that are blurry, distorted, or have abnormal features;

[0067] S16, merge the filtered samples with the original samples belonging to the scarce category to form an amplified pavement disease sample dataset.

[0068] In this embodiment, generative adversarial network refers to a family of deep learning models that includes a generator and a discriminator and generates images through an adversarial game mechanism. Its specific architecture can include various specific implementations such as style-based generative adversarial network and deep convolutional generative adversarial network, as long as it can generate realistic similar target generation regions based on the input scarce category slices.

[0069] When training a generative adversarial network (GAN), due to the extremely limited number of real samples in rare categories, the discriminator often quickly memorizes the features of these limited real samples, leading to overfitting. This, in turn, prevents the generator from obtaining effective gradient feedback, ultimately causing pattern collapse and resulting in the generation of only single-morphological samples lacking diversity. To overcome the persistent problem of training with few samples, this embodiment introduces an adaptive data augmentation strategy into the GAN. The core mechanism of this strategy is that during training, data augmentation operations such as geometric transformations and color transformations are dynamically applied to the real images input to the discriminator and the generated images. For example, random rotation and color jitter are used to artificially expand the sample diversity seen by the discriminator, effectively preventing the discriminator from forcibly memorizing limited real samples, maintaining a healthy adversarial balance between the discriminator and the generator, and significantly improving the morphological diversity and detail clarity when generating rare samples.

[0070] It should be understood that the augmentation strength of the adaptive data augmentation strategy is dynamically adjusted with the training process. When the risk of discriminator overfitting is high, the augmentation strength is automatically increased, and when the generation quality tends to stabilize, the augmentation strength is automatically decreased.

[0071] After generating the initial disease samples added by the adversarial network output, the quality screening process includes at least one of the following:

[0072] Remove samples with blurred edges by calculating image gradient and local variance;

[0073] Samples with distorted lighting are removed by detecting areas of abnormal color distribution.

[0074] Samples with abnormal structural morphology are removed by comparing category feature templates.

[0075] S20, Based on the statistical distribution characteristics of pavement distress categories in the amplified pavement distress sample dataset, determine the spatial positional relationship of multiple target distress sample categories in the pavement background image, wherein the statistical distribution characteristics include the coexistence probability and spatial distance distribution of the target distress sample categories, and the spatial positional relationship is determined based on the coexistence probability and the spatial distance distribution;

[0076] In this embodiment, statistical distribution characteristics refer to quantitative indicators extracted from a large amount of real sample data that reflect the objective occurrence patterns and spatial relationships between target objects. In real-world scenarios, certain types of target objects tend to appear together, and their distances and relative orientations show a clear concentration trend, rather than being randomly scattered without any pattern. If multiple target objects are randomly assigned to positions in the background image, it is easy to produce layouts that violate physical common sense or actual scene characteristics. For example, objects that should be far apart are forcibly overlapped, or objects that should be related at a specific angle are randomly pieced together, thus compromising the realism of the macroscopic scene. Therefore, this embodiment introduces statistical distribution characteristics as constraints on spatial positional relationships to avoid layout distortion caused by random stacking.

[0077] Furthermore, relying solely on simple category frequency statistics to arrange objects can easily lead to object combinations that violate physical principles, such as forcibly piecing together objects that would never normally appear together in the same scene. Therefore, this embodiment further introduces coexistence probability and spatial distance distribution to avoid such unreasonable object piecing together.

[0078] The coexistence probability reflects the likelihood of different categories of target objects appearing simultaneously in real samples. Only category combinations with a coexistence probability reaching a certain threshold are allowed as candidate combinations to be synthesized. The spatial distance distribution further constrains the relative positions between these objects after determining reasonable category combinations. It specifies the physical distance and orientation relationship that objects should maintain, avoiding objects that are too far apart and appear isolated or too close and cause meaningless overlap.

[0079] Further and optionally, the spatial distance distribution includes the Euclidean distance between the center points of the target objects, the degree of overlap between the target generation regions, and the relative angle between the target generation regions. The steps for determining the spatial positional relationship include:

[0080] The combination of target disease categories to be synthesized is determined based on the coexistence probability.

[0081] Based on the Euclidean distance, the degree of overlap, and the relative angle, the relative positional relationship of each target disease category in the target disease category combination in the road surface background image is determined.

[0082] Specifically, for how to determine the relative position relationship, three types of constraint features, namely Euclidean distance, overlap degree, and relative angle, can be fused for comprehensive determination. First, extract the center coordinates of the two disease targets, and measure their spatial proximity by the Euclidean distance between the two. Second, use the intersection over union (IoU) to quantify the overlap degree: if IoU≈0, it indicates that there is no overlap between the two; if 0 < IoU < τ (τ is a preset overlap threshold), it indicates that the two intersect locally; if IoU is close to 1, it indicates that the two almost completely coincide or contain each other. Finally, the relative angle of the center connection line is used to characterize the azimuth distribution relationship. For example, an angle of 0° means that one disease is directly to the right of the other, 90° means directly above, and the remaining angles follow this rule.

[0083] S30. Determine the complexity of the target generation region of the target disease sample category, and determine the corresponding target fusion strategy according to the complexity.

[0084] In this embodiment, the complexity refers to the quantization value of the texture richness and the sharpness of the edge structure in the visual presentation of the target disease sample category in the target generation region.

[0085] In the field of image fusion, high-complexity regions often contain a large number of irregular textures and sharp edges. If the fusion is excessive, these key details are likely to be blurred and lost. If the fusion is insufficient, there will be a rigid stitching boundary left. The textures in low-complexity regions are smooth but the edges are blurred. The difficulty of their fusion lies in the seamless transition of light and color. If too strong structure-preserving fusion is applied, it will instead introduce artifacts or noise. Therefore, the traditional method of using a fixed single fusion strategy to process all regions cannot take into account both of the above two fusion situations simultaneously.

[0086] Therefore, in this embodiment, by dividing the complexity into different preset intervals and matching a suitable target fusion strategy for each interval, the fusion algorithm can perform image fusion according to the visual characteristics of the region itself.

[0087] Specifically and optionally, S30 specifically includes:

[0088] S31. Obtain the edge density and texture variance of the target generation region of the target disease sample category.

[0089] S32. Take the average of the sum of the edge density and the texture variance and normalize it to obtain the complexity.

[0090] S33. Determine the target fusion strategy according to the preset interval where the complexity is located.

[0091] Further and optionally, the preset interval includes a high complexity interval, a medium complexity interval, and a low complexity interval. In some optional implementations, when the complexity C is greater than 0.5, it belongs to the high complexity interval; when the complexity C is greater than 0.3 and less than 0.5, it belongs to the medium complexity interval; and when the complexity C is less than 0.3, it belongs to the low complexity interval.

[0092] S33 includes:

[0093] S331, when the complexity is in the high complexity range, the selected target fusion strategy is multi-scale Poisson fusion or texture-aware fusion.

[0094] Further and optionally, multi-scale Poisson fusion includes the following steps:

[0095] The target generation region and the road background image are decomposed into a multi-scale pyramid; the gradient field of the target generation region at each scale in the multi-scale pyramid is calculated; based on the gradient field and the pixel value of the background region on the fusion boundary, the Poisson equation of the corresponding scale is solved; and the solution results of each scale are used for pyramid reconstruction and fusion based on the Poisson equation.

[0096] The texture-aware fusion includes the following steps:

[0097] Determine the target generation region of the target disease sample category and the texture features of the road surface background image; adjust the texture attributes of the target generation region according to the texture features to match the road surface background image;

[0098] It's worth noting that the chosen fusion strategies, multi-scale Poisson fusion and texture-aware fusion, are both highly effective at preserving structure and transferring detail. Specifically, multi-scale Poisson fusion, with its gradient domain-based solution mechanism, can seamlessly integrate rich and sharp edge details from the target generation region into the background while maintaining the continuity of the background gradient field, thus avoiding detail blurring. Texture-aware fusion, by explicitly extracting and matching LBP features, gradient magnitude, and roughness, ensures that the texture undulations of highly complex regions blend naturally with the surrounding background, eliminating the sense of texture discontinuity.

[0099] S332, when the complexity is in the medium complexity range and the area of ​​the target generation region is greater than the preset area threshold, the selected target fusion strategy is multi-scale Poisson fusion.

[0100] It should be noted that although medium-complexity regions are not as sharp as high-complexity regions in terms of detail, if their area is large, it means that they occupy a significant visual area of ​​the synthesized image. In this case, if a simple color transition fusion is used, it is easy to produce a patchy appearance with inconsistent lighting over a large area. Therefore, this embodiment still schedules multi-scale Poisson fusion under this condition, using its gradient domain solution capability to ensure global consistency in structure and lighting between large medium-complexity regions and the background.

[0101] S333, when the complexity is in the low complexity range, the selected target fusion strategy is illumination consistency fusion or progressive Alpha fusion.

[0102] Further and optionally, the illumination uniformity fusion includes the following steps:

[0103] In the LAB color space, the brightness channel of the target generation region is adjusted to match the lighting conditions of the road background image. The color of the target generation region is matched with the color distribution of the road background image. A multi-layer fusion strategy is used to fuse the adjusted target generation region into the road background image.

[0104] The progressive Alpha fusion includes the following steps:

[0105] A directed distance field is constructed for the target generation region. Based on this distance field, a nonlinear transparency function is called to generate a progressive mask. The progressive mask is then used to progressively fuse the target generation region and the road background image.

[0106] It's important to note that both consistent lighting blending and progressive alpha blending are suitable for low-complexity regions because these areas lack the texture details worth preserving. The core focus of their blending is the smooth transition between color and brightness. Applying strong structural blending methods like Poisson blending can introduce artifacts due to gradient field instability. Consistent lighting blending involves separating brightness and color in the LAB space, globally aligning the brightness channel, mapping the color distribution, and then gradually blending through multi-layered strategies to ensure the naturalness of lighting and color in low-complexity regions meets our requirements. Progressive alpha blending, on the other hand, constructs a progressive mask that is completely opaque at the center and gradually becomes transparent towards the edges. Combined with edge smoothing and basic attribute matching, it achieves a smooth transition from the center to the edges in low-complexity regions, thus eliminating hard edges.

[0107] S40, according to the spatial relationship, the target fusion strategy is used to fuse the target disease sample category into the road surface background image to generate a road surface disease image.

[0108] In this step, the spatial positional relationship determined in S20 and the target fusion strategy determined in S30 are invoked to generate each target generation region at the corresponding image coordinate position of the road background image; then, for each placed target generation region, the target fusion strategy performs boundary and texture fusion processing to complete the generation of the road damage image.

[0109] Further and optionally, the generated road surface defect image may include at least one environmental effect, including different lighting conditions, weather conditions, and road surface stains.

[0110] Specifically, the lighting effect can be achieved using a lighting simulation algorithm to simulate the road surface effect under different light intensities. This involves adjusting the image's brightness, contrast, and color temperature to ensure that the features of the affected area are clearly displayed under varying lighting conditions. For example, when simulating direct sunlight, the brightness is adjusted to 1.25, the contrast to 1.15, the saturation to 1.08, the hue shift to 3, and shadows are added with a density of 0.3.

[0111] For weather effects, rain is achieved by adding raindrop textures and adjusting image saturation and contrast, while fog is achieved by adding fog textures and reducing image sharpness. For example, when simulating rain, the brightness is adjusted to 0.65, the contrast to 0.88, the saturation to 1.12, and the hue shift to -12, thereby adding wetness, reflections, water accumulation, and raindrop effects.

[0112] For the road stain effect, stain textures are randomly generated and overlaid on the road background and damaged areas. For example, when simulating a road surface with stains, the brightness is adjusted to 0.8, the contrast to 0.9, the saturation to 0.8, an oil stain effect is added, and the opacity of the stains is set to 0.65.

[0113] For example, refer to Figure 2 The example images of road surface defects generated by the method based on statistical distribution feature constraints and multi-strategy fusion proposed in this application are shown. It can be seen that the generated images are characterized by: the morphology, texture, color, and spatial distribution of defects highly consistent with the statistical characteristics of real road surface defects; natural transition between the background and defect areas; and the ability to cover various defect types such as cracks, potholes, and spalling, resulting in a realistic overall visual effect. Compared to existing image generation methods, this method effectively preserves the details of the real road surface background, avoids irrelevant texture distortion, and generates defect morphologies that better match actual engineering scenarios. Furthermore, the synthetic data domain offset is small and highly controllable, accurately addressing the problem of sample imbalance. Simultaneously, it has low computational cost and high generation efficiency, demonstrating significant advantages in expanding defect datasets and improving the generalization ability of detection models.

[0114] In the technical solution provided in this embodiment, the sample data of different road surface defects are balanced by augmenting the sample data of scarce categories. Then, based on the statistical distribution characteristics of the selected target defect sample category in the sample data, the spatial position relationship in the roadside background image is determined, so that the multiple objects generated in the synthetic image follow the coexistence probability and spatial distance distribution law in the real sample. Next, the corresponding target fusion strategy is determined based on the complexity of the target generation region, so that the fusion algorithm can match the appropriate fusion method according to the complexity of the target object region itself, thereby avoiding the splicing traces and inconsistencies in lighting texture caused by a single fusion strategy.

[0115] Second Embodiment

[0116] Based on the first embodiment, this embodiment provides a method for quality assessment of generated pavement defect images. Multi-dimensional quantitative indicators are used to measure the difference between the generated image and the real image. The main assessment indicators include, but are not limited to:

[0117] Structural similarity (SSIM) measures the brightness, contrast, and structure of two images. It more closely approximates human visual perception and more accurately reflects the quality of generated images. The definition of SSIM can be expressed as follows:

[0118]

[0119] in, and These are images and The average grayscale value is obtained by averaging the values ​​of all pixels and represents the brightness level of the image. and These are images and The standard deviation of gray levels represents the contrast of the image; It is an image and The gray covariance reflects the structural similarity between the two.

[0120] , ,generally , , It is the dynamic range of pixel values.

[0121] Fréchet Inception Distance (FID) is a commonly used metric to measure the similarity of the distribution of generated and real images in feature space. This metric extracts image features based on a pre-trained Inception network and calculates the Fréchet distance between the real and generated data in feature space. The definition of FID can be expressed as follows:

[0122]

[0123] in, For the Euclidean norm, The trace of the matrix, The FID value is the square root of the matrix (which can be achieved through singular value decomposition). The smaller the FID value, the closer the generated image is to the real image in terms of feature distribution, and the higher the generation quality.

[0124] Class balance is used to detect whether some classes are over-generated or severely missing. Assumptions Indicates the first The number of generated samples for each category Let the set of sample sizes for all classes be represented by the following formula:

[0125]

[0126] in, and represents the standard deviation and mean of the sample size, respectively. The class balance ranges from [0,1]. The closer the value is to 1, the more balanced the sample size of each class is; conversely, if the value is small, it indicates that there is a significant class bias.

[0127] Spatial rationality is used to evaluate whether the spatial distribution of diseased areas in the generated image conforms to physical laws and the characteristics of the actual scene. Ideally, the diseased areas should be distributed in reasonable locations, and their shape and size should have a certain naturalness. Assumptions For the first In the nth sample Spatial attributes of a disease instance (such as center coordinates, area, aspect ratio, etc.). and These are its mean and standard deviation, respectively. Given the set of all disease instances, the definition of spatial rationality can be expressed as follows:

[0128]

[0129] in, This is an indicator function that takes the value 1 when the condition is met, and 0 otherwise. This indicator reflects the proportion of the spatial attributes of the disease falling within the normal range. The closer the value is to 1, the more reasonable the spatial distribution of the generated samples.

[0130] The first two metrics (SSIM and FID) are used to evaluate the image quality generated by StyleGAN, while the latter two metrics (Balance and Spatial) are used to evaluate the rationality of the category and spatial distribution of the final generated pavement distress images.

[0131] In addition, refer to Figure 3 This embodiment also proposes a computer system for generating pavement distress images based on statistical distribution feature constraints and multi-strategy fusion, characterized in that the computer system includes:

[0132] The small sample disease category data augmentation module 100 is used to augment the data of various target disease samples belonging to scarce categories in the road surface disease sample dataset. The target disease samples of scarce categories are divided based on the proportion of each disease category in the road surface disease sample dataset.

[0133] The multi-disease category combination and layout module 200 is used to determine the spatial position relationship of multiple target disease sample categories in a road background image based on the statistical distribution characteristics of road disease categories in the amplified road disease sample dataset. The statistical distribution characteristics include the coexistence probability and spatial distance distribution of the target disease sample categories, and the spatial position relationship is determined based on the coexistence probability and the spatial distance distribution.

[0134] The fusion strategy selection module 300 is used to determine the complexity of the target generation region of the target disease sample category, and determine the corresponding target fusion strategy based on the complexity.

[0135] The fusion module 400 is used to fuse the target disease sample category into the road surface background image according to the spatial position relationship and the target fusion strategy to generate a road surface disease image.

[0136] Furthermore, those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in a computer system to implement the process steps of the embodiments of the above methods.

[0137] Therefore, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the various steps of the road surface distress image generation method based on statistical distribution feature constraints and multi-strategy fusion as described in the above embodiments.

[0138] The computer-readable storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0139] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0140] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for generating pavement distress images based on statistical distribution feature constraints and multi-strategy fusion, characterized in that, The method includes the following steps: Data augmentation is performed on target disease samples belonging to scarce categories in the road surface disease sample dataset. The scarce category of target disease samples is divided based on the proportion of each disease category in the road surface disease sample dataset. Based on the statistical distribution characteristics of pavement distress categories in the amplified pavement distress sample dataset, the spatial positional relationship of multiple target distress sample categories in the pavement background image is determined. The statistical distribution characteristics include the coexistence probability and spatial distance distribution of the target distress sample categories, and the spatial positional relationship is determined based on the coexistence probability and the spatial distance distribution. Determine the complexity of the target generation region for the target disease sample category, and determine the corresponding target fusion strategy based on the complexity; Based on the spatial relationship, the target fusion strategy is used to fuse the target disease sample categories into the road surface background image to generate a road surface disease image.

2. The method for generating pavement distress images based on statistical distribution feature constraints and multi-strategy fusion as described in claim 1, characterized in that, The spatial distance distribution includes the Euclidean distance between the center points of the target objects, the degree of overlap between the target generation regions, and the relative angles between the target generation regions. The steps for determining the spatial positional relationship include: The combination of target disease categories to be synthesized is determined based on the coexistence probability. Based on the Euclidean distance, the degree of overlap, and the relative angle, the relative positional relationship of each target disease category in the target disease category combination in the road surface background image is determined.

3. The method for generating pavement distress images based on statistical distribution feature constraints and multi-strategy fusion as described in claim 1, characterized in that, The process of determining the complexity of the target generation region for the target disease sample category, and determining the corresponding target fusion strategy based on the complexity of the target generation region, includes: Obtain the edge density and texture variance of the target generation region for the target disease sample category; The complexity is obtained by averaging the sum of the edge density and the texture variance and then normalizing it. The target fusion strategy is determined based on the preset range in which the complexity falls.

4. The method for generating pavement distress images based on statistical distribution feature constraints and multi-strategy fusion as described in claim 3, characterized in that, The preset interval includes a high complexity interval, a medium complexity interval, and a low complexity interval. Determining the target fusion strategy based on the preset interval in which the complexity falls includes: When the complexity falls within the high complexity range, the selected target fusion strategy is multi-scale Poisson fusion or texture-aware fusion. When the complexity is in the medium complexity range and the area of ​​the target generation region is greater than the preset area threshold, the selected target fusion strategy is multi-scale Poisson fusion. When the complexity falls within the low complexity range, the selected target fusion strategy is either illumination-consistent fusion or progressive Alpha fusion.

5. The method for generating pavement distress images based on statistical distribution feature constraints and multi-strategy fusion as described in claim 4, characterized in that, The multi-scale Poisson fusion includes the following steps: The target generation region and the road background image are decomposed into a multi-scale pyramid; the gradient field of the target generation region at each scale in the multi-scale pyramid is calculated; based on the gradient field and the pixel value of the background region on the fusion boundary, the Poisson equation of the corresponding scale is solved; and the solution results of each scale are used for pyramid reconstruction and fusion based on the Poisson equation. The texture-aware fusion includes the following steps: Determine the target generation region of the target disease sample category and the texture features of the road surface background image; adjust the texture attributes of the target generation region according to the texture features to match the road surface background image; The illumination uniformity fusion includes the following steps: In the LAB color space, the brightness channel of the target generation region is adjusted to match the lighting conditions of the road background image. The color of the target generation region is matched with the color distribution of the road background image. A multi-layer fusion strategy is used to fuse the adjusted target generation region into the road background image. The progressive Alpha fusion includes the following steps: A directed distance field is constructed for the target generation region. Based on this distance field, a nonlinear transparency function is called to generate a progressive mask. The progressive mask is then used to progressively fuse the target generation region and the road background image.

6. The method for generating pavement distress images based on statistical distribution feature constraints and multi-strategy fusion as described in claim 1, characterized in that, The data type of the pavement defect sample dataset includes scarce categories and sufficient categories, and the steps for dividing the scarce categories and sufficient categories include: Determine the ratio of the number of samples for each type of pavement distress to the total number of samples in the pavement distress sample dataset; Pavement distress sample types with ratios less than a preset threshold are identified as scarce categories. Otherwise, it is determined to be the sufficient category, wherein the pavement defect sample data corresponding to the sufficient category is not expanded.

7. The method for generating pavement distress images based on statistical distribution feature constraints and multi-strategy fusion as described in claim 1 or 6, characterized in that, The steps for data augmentation of target defect samples belonging to scarce categories in a road surface defect sample dataset include: An initial generated region consistent with the scarce category is generated by a generative adversarial network, which employs an adaptive data augmentation strategy to prevent the discriminator from overfitting. The initial generated region is subjected to quality screening to remove samples that are blurry, distorted, or have abnormal features; The filtered samples are merged with the original samples belonging to the scarce category to form an amplified pavement disease sample dataset.

8. The method for generating pavement distress images based on statistical distribution feature constraints and multi-strategy fusion as described in claim 1, characterized in that, The generated road surface defect image is supplemented with at least one environmental effect, including different lighting conditions, weather conditions, and road surface stains.

9. A computer system for implementing the pavement distress image generation method based on statistical distribution feature constraints and multi-strategy fusion as described in any one of claims 1 to 8, characterized in that, The computer system includes: The small sample disease category data augmentation module is used to augment the target disease samples belonging to scarce categories in the road surface disease sample dataset. The target disease samples of scarce categories are divided based on the proportion of each disease category in the road surface disease sample dataset. A multi-disease category combination and layout module is used to determine the spatial positional relationship of multiple target disease sample categories in a road background image based on the statistical distribution characteristics of road disease categories in the amplified road disease sample dataset. The statistical distribution characteristics include the coexistence probability and spatial distance distribution of the target disease sample categories, and the spatial positional relationship is determined based on the coexistence probability and the spatial distance distribution. The fusion strategy selection module is used to determine the complexity of the target generation region of the target disease sample category, and determine the corresponding target fusion strategy based on the complexity. The fusion module is used to fuse the target disease sample category into the road surface background image according to the spatial position relationship and the target fusion strategy to generate a road surface disease image.

Images