A method and system for road maintenance discrimination

Abstract

This invention provides a road maintenance discrimination method and system. The method includes: converting an initial road surface image into an enhanced illumination component and an enhanced reflection component to obtain an enhanced road surface image; obtaining a texture feature map corresponding to the enhanced road surface image, and then obtaining a global feature sequence; obtaining a determined disease category corresponding to the global feature sequence; obtaining a contour image based on the enhanced road surface image; obtaining a semantic feature tensor and a contour feature tensor through the enhanced road surface image and the contour image, and then obtaining a binarized segmentation map including the disease area; obtaining a road surface score based on the determined disease category and the binarized segmentation map to determine whether road maintenance is required. By obtaining the global feature sequence, the accuracy of disease category recognition is improved, and the combination with the enhanced road surface image enables accurate detection of early-stage diseases. The combination of the semantic feature tensor and the contour feature tensor effectively suppresses false detections caused by non-disease edges, further improving the detection accuracy of diseases.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for determining road maintenance. Background Technology

[0002] Highways are the core infrastructure of modern transportation systems, and their quality directly affects transportation efficiency and driving safety. As an important carrier of people's lives, highways drive the economic development of the entire city, and their importance in urban construction and development has received increasing attention.

[0003] As highways age, their quality gradually declines due to long-term effects from vehicle loads, climate change, and the natural environment, leading to defects such as cracks. If not maintained in time, these defects will expand, accelerating highway aging, increasing maintenance costs, and potentially causing traffic accidents and threatening public safety. Therefore, timely detection and repair of road surface defects are of great significance in preventing highway safety accidents.

[0004] Currently, road surface defects are generally identified through road surface defect detection, and then road maintenance is carried out accordingly. However, the defect identification process generally uses sensor detection, which relies on sensors such as accelerometers and strain gauges installed on vehicles to collect signals. While it can cover a wide range, its detection accuracy is insufficient, and it is difficult to identify early-stage defects. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a road maintenance identification method and system, which solves the technical problems of insufficient detection accuracy and difficulty in identifying early-stage defects when relying on sensors for pre-maintenance defect detection in the road maintenance identification process.

[0006] To achieve the above objectives, in a first aspect, embodiments of this application provide a road maintenance determination method, comprising the following steps:

[0007] An initial road surface image is acquired, and the initial road surface image is converted into an enhanced illumination component and an enhanced reflection component. An enhanced road surface image is then obtained through the enhanced illumination component and the enhanced reflection component.

[0008] Obtain the texture feature map corresponding to the enhanced road surface image, convert the texture feature map into a global feature sequence, and obtain the determination disease category corresponding to the global feature sequence;

[0009] Edge detection is performed on the enhanced road surface image to obtain a contour image. Semantic feature tensors and contour feature tensors are obtained based on the enhanced road surface image and the contour image, respectively. A binarized segmentation map is obtained through the semantic feature tensor and the contour feature tensor, and the binarized segmentation map includes the diseased area.

[0010] Based on the determined disease category and the binarized segmentation map, a road surface score is obtained. The road surface score is then compared with a score threshold to determine whether road maintenance is required.

[0011] Furthermore, the step of converting the initial road surface image into an enhanced illumination component and an enhanced reflection component includes:

[0012] An initial optimization model is constructed, and a training input set is obtained, which includes training road surface images. The training input set is used as the input value of the initial optimization model, and the initial optimization model outputs training illumination output components and training reflection output components.

[0013] Based on the training illumination output component and the training reflection output component, a reconstruction loss function, a reflection component constraint loss function, and an illumination component constraint loss function are constructed respectively, and a total loss function is constructed through the reconstruction loss function, the reflection component constraint loss function, and the illumination component constraint loss function.

[0014] The initial optimization model is updated to the final optimization model using the total loss function, and the initial road surface image is used as the input value of the final optimization model to obtain the enhanced illumination component and the enhanced reflection component through the final optimization model.

[0015] Furthermore, the expression for the reconstruction loss function is:

[0016] ,

[0017] in, Represents the reconstruction loss function. Represents the training road surface image, This represents the training reflection output component. This represents the training illumination output component. This indicates element-wise multiplication. Represents the L1 norm;

[0018] The expression for the reflection component constraint loss function is:

[0019] ,

[0020] in, Represents the reflection component constraint loss function. The V channel represents the training reflection output component. This represents the V channel of the training road surface image. Represents the HSV color space. Indicates channel extraction operation. This indicates an adaptive histogram equalization operation. Indicates the noise suppression coefficient. This represents the image gradient of the training reflection output component;

[0021] The expression for the illumination component constraint loss function is:

[0022] ,

[0023] in, Represents the illumination component constraint loss function. This represents the image gradient of the training illumination output component. Represents the coupling coefficient. This represents an exponential function.

[0024] Furthermore, the step of obtaining the texture feature map corresponding to the enhanced road surface image and converting the texture feature map into a global feature sequence includes:

[0025] The texture feature map of the enhanced road surface image is extracted using a CNN network, and the texture feature map includes several grid regions;

[0026] In the spatial dimension, the texture feature map is flattened into a texture feature sequence, which includes several texture feature vectors corresponding to the grid region. Position encoding is added to the texture feature vectors to obtain several local feature vectors.

[0027] The local feature vectors are updated to global feature vectors through a multi-head attention mechanism, and several of the global feature vectors constitute a global feature sequence.

[0028] Furthermore, the formula for obtaining the location code is:

[0029] ,

[0030] in, This represents the value of the position code in the 2i-th dimension. This represents the position code value in the (2i+1)th dimension. This represents the position index corresponding to the grid region, and 0 ≤ < ,in Indicates the total number of grid regions. Represents a dimension index, and 0 ≤ < , This represents the total dimension of the texture feature vector.

[0031] Furthermore, the step of obtaining the disease category corresponding to the global feature sequence includes:

[0032] Dimensional equalization is performed on all the global feature vectors to obtain aggregate vectors;

[0033] The aggregated vector is linearly transformed to convert it into a raw category score dataset, which includes several category scores that correspond to preset disease categories.

[0034] Based on the category score, the disease probability corresponding to the preset disease category is obtained, and the preset disease category with the highest disease probability is selected as the disease category to be measured.

[0035] Furthermore, the formula for obtaining the category score is:

[0036] ,

[0037] in, This represents the category score corresponding to the i-th preset disease category. Represents an aggregate vector. Represents the weight matrix. This indicates the bias term.

[0038] Furthermore, the step of obtaining the binarized segmentation map through the semantic feature tensor and the contour feature tensor includes:

[0039] The semantic feature tensor and the contour feature tensor are concatenated into a combined tensor, and a spatial attention map is obtained based on the combined tensor.

[0040] The semantic feature tensor is weighted and modulated using the spatial attention map to obtain an enhanced feature tensor. The enhanced feature tensor is then upsampled to obtain a single-channel feature map with the same size as the enhanced road surface image.

[0041] The original pixel value of each pixel in the single-channel feature map is mapped to a pixel probability. The pixel probability is compared with a probability threshold to determine the final pixel value of the pixel. The final pixel value is 0 or 1, so as to convert the single-channel feature map into a binary segmentation map.

[0042] Furthermore, the step of obtaining a pavement score based on the determined disease category and the binarized segmentation map includes:

[0043] Obtain the disease weight corresponding to the measured disease category, and obtain the disease area of ​​the disease region;

[0044] The pavement score is obtained based on the defect weight and the defect area.

[0045] Secondly, embodiments of this application provide a road maintenance discrimination system, applied to the road maintenance discrimination method described in the first aspect above, the system comprising:

[0046] The acquisition module is used to acquire an initial road surface image, convert the initial road surface image into an enhanced illumination component and an enhanced reflection component, and acquire an enhanced road surface image through the enhanced illumination component and the enhanced reflection component.

[0047] The analysis module is used to obtain a texture feature map corresponding to the enhanced pavement image, convert the texture feature map into a global feature sequence, and obtain the measured disease category corresponding to the global feature sequence;

[0048] An extraction module is used to perform edge detection on the enhanced road surface image to obtain a contour image, obtain a semantic feature tensor and a contour feature tensor based on the enhanced road surface image and the contour image respectively, and obtain a binarized segmentation map through the semantic feature tensor and the contour feature tensor, wherein the binarized segmentation map includes the diseased area.

[0049] The execution module is used to obtain a pavement score based on the determined disease category and the binarized segmentation map, and compare the pavement score with a score threshold to determine whether road maintenance is required.

[0050] Thirdly, embodiments of this application provide a computer, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the road maintenance determination method as described in the first aspect above.

[0051] Fourthly, embodiments of this application provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the road maintenance determination method as described in the first aspect above.

[0052] Compared with existing technologies, the beneficial effects of this invention are as follows: By acquiring the enhanced illumination component and the enhanced reflection component, an enhanced pavement image with uniform illumination, prominent details, and controlled noise is obtained, providing a stable and highly separable input for subsequent processing, laying the foundation for detecting early-stage, low-contrast defects; By acquiring the global feature sequence, the defect category is determined based on the understanding of the overall morphology and spatial layout of the defects, improving the accuracy of identifying defect categories with complex morphology and discontinuous structure, and achieving accurate detection of early-stage defects by combining the enhanced pavement image; By combining the semantic feature tensor and the contour feature tensor, false detections caused by non-defect edges such as pavement markings, stone textures, and water stains are effectively suppressed, significantly improving the signal-to-noise ratio of the segmentation results and further improving the detection accuracy of defects; By acquiring the pavement score, a complete closed loop for road maintenance discrimination is achieved, transforming post-maintenance into predictive maintenance. Attached Figure Description

[0053] Figure 1 This is a flowchart of the road maintenance determination method in the first embodiment of the present invention;

[0054] Figure 2 This is a structural block diagram of the road maintenance discrimination system in the second embodiment of the present invention;

[0055] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0056] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0057] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0059] Please see Figure 1 The road maintenance determination method provided in the first embodiment of the present invention includes the following steps:

[0060] S10: Obtain an initial road surface image, convert the initial road surface image into an enhanced illumination component and an enhanced reflection component, and obtain an enhanced road surface image through the enhanced illumination component and the enhanced reflection component;

[0061] In this embodiment, aerial photography by a drone is used to acquire images. These images are then divided into grids to form several smaller initial road surface images, thus avoiding the possibility of multiple defects appearing in the initial road surface images. The initial road surface images are affected by the shooting environment (outdoor lighting is affected by weather, time, and obstructions), causing the grayscale difference between defects and the road surface to become extremely low in dark or bright areas, making the defects hidden in the background and unable to be effectively detected. Furthermore, the signal strength of early defects is much lower than that of normal road surface texture and noise. In the initial road surface images, the signal-to-noise ratio of early defects is too low to be identified. By acquiring the enhanced road surface images, subsequent accurate identification of defects can be facilitated.

[0062] Step S10 includes:

[0063] S110: Construct an initial optimization model and obtain a training input set, the training input set including training road surface images, use the training input set as the input value of the initial optimization model, and output training illumination output components and training reflection output components through the initial optimization model;

[0064] S120: Construct a reconstruction loss function, a reflection component constraint loss function, and an illumination component constraint loss function based on the training illumination output component and the training reflection output component, respectively, and construct a total loss function through the reconstruction loss function, the reflection component constraint loss function, and the illumination component constraint loss function;

[0065] In this embodiment, the initial optimization model is an unsupervised deep learning network model. Since it is impossible to obtain absolutely correct ground truth values ​​for illumination and reflection corresponding to the training road surface image, a reconstruction loss function, a reflection component constraint loss function, and an illumination component constraint loss function are set to measure the accuracy of the training illumination output component and the training reflection output component. These three functions correspond to physical rules, the expected attributes of the reflection map, and the expected attributes of the illumination map, respectively.

[0066] The expression for the reconstruction loss function is:

[0067] ,

[0068] in, Represents the reconstruction loss function. Represents the training road surface image, This represents the training reflection output component. This represents the training illumination output component. This indicates element-wise multiplication. The L1 norm is represented; the reconstruction loss function makes the training illumination output component and the training reflection output component inversely related to each other, ensuring that their aggregation is close to the training road surface image, and ensuring the basic correctness of the decomposition process.

[0069] The expression for the reflection component constraint loss function is:

[0070] ,

[0071] in, Represents the reflection component constraint loss function. The V channel represents the training reflection output component. This represents the V channel of the training road surface image. Represents the HSV color space. Indicates channel extraction operation. This indicates an adaptive histogram equalization operation. Indicates the noise suppression coefficient. The image gradient represents the training reflection output component; by introducing an adaptive histogram equalization operation to process the V channel of the training road surface image, it is used as a reference template for detail enhancement, thereby gradually reducing the gradient of the reflection map, reflecting the prior condition that the reflection map should suppress noise and unnecessary texture abrupt changes.

[0072] The expression for the illumination component constraint loss function is:

[0073] ,

[0074] in, Represents the illumination component constraint loss function. This represents the image gradient of the training illumination output component. Represents the coupling coefficient. The exponential function represents the lighting component constraint loss function, which allows abrupt changes in lighting at object edges, while forcing smooth lighting in flat areas, thereby ensuring that the trained lighting output components conform to the physical condition that lighting is piecewise smooth.

[0075] The expression for the total loss function is:

[0076] ,

[0077] in, Represents the total loss function. , All of these represent hyperparameters. In this embodiment, The value is 0.5. The value is 0.1;

[0078] S130: The initial optimization model is updated to the final optimization model through the total loss function, and the initial road surface image is used as the input value of the final optimization model to obtain the enhanced illumination component and the enhanced reflection component through the final optimization model;

[0079] After obtaining the enhanced illumination component and the enhanced reflection component, the enhanced road surface image is obtained by multiplying them element-wise. The enhanced road surface image has the characteristics of balanced brightness and prominent details. The entire optimization process of the final optimization model only requires the input of the training road surface image, without any "standard enhancement result" as a label. Its supervision signal comes from the total loss function formed by the reconstruction loss function, the reflection component constraint loss function, and the illumination component constraint loss function, which reduces the data threshold and cost of model training. It achieves targeted, adaptive, and high-fidelity enhancement of the details of defects in the image, while effectively suppressing noise and illumination interference, providing an accurate basis for subsequent defect identification.

[0080] S20: Obtain the texture feature map corresponding to the enhanced road surface image, convert the texture feature map into a global feature sequence, and obtain the disease category corresponding to the global feature sequence;

[0081] Step S20 includes:

[0082] S210: Extract the texture feature map of the enhanced road surface image through a CNN network, wherein the texture feature map includes several grid regions;

[0083] Specifically, the enhanced road surface image (e.g., 512*512 pixels) is input into the CNN network. First, preliminary feature extraction is performed using a 7*7 convolutional kernel to obtain several output feature maps. Each output feature map corresponds to a local region centered on a specific pixel in the enhanced road surface image. The size of this local region is equal to the size of the convolutional kernel (7*7). Then, max pooling is used to reduce the resolution of the output feature maps to preserve salient features. For example, a 2*2 pooling window halves the size of the output feature maps to reduce computational load and simultaneously enhance the translational invariance of texture. The pooled output feature maps are simultaneously input into a series of residual blocks for deep processing. Shallow residual blocks learn to capture simple local texture patterns (such as crack edges and line segments), while deeper residual blocks learn to capture more complex, global texture patterns (such as the interlacing patterns of cracks), thereby generating the texture feature map (taking a sampling rate of 32x as an example, the structure of the texture feature map is 16*16*1024). Based on this, the texture feature map has 16*16=256 grid regions, and each grid region has a feature dimension of 1024. CNN networks are widely used and will not be elaborated upon here.

[0084] S220: Flatten the texture feature map into a texture feature sequence in the spatial dimension. The texture feature sequence includes several texture feature vectors corresponding to the grid region. Add position encoding to the texture feature vectors to obtain several local feature vectors.

[0085] Specifically, a position index is generated based on the positional relationship of the grid regions in the texture feature map. Assuming the texture feature map comprises 3*3=9 grids, each grid has a feature dimension of 4, for the grid (1,1), its position index... If the value is 0, a 4-dimensional location code needs to be generated based on this location index.

[0086] The formula for obtaining the location code is:

[0087] ,

[0088] in, This represents the value of the position code in the 2i-th dimension. This represents the position code value in the (2i+1)th dimension. This represents the position index corresponding to the grid region, and 0 ≤ < ,in Indicates the total number of grid regions. Represents a dimension index, and 0 ≤ < , This represents the total dimension of the texture feature vector.

[0089] When i is 0, PE(0,0)=0, PE(0,1)=1; when i is 1, PE(0,2)=0, PE(0,3)=1. The position code is [0,1,0,1]. Assuming the texture feature vector corresponding to this grid region is [0.5,0.2,0.8,0.1], the two are summed channel by channel to generate the local feature vector [0.5,1.2,0.8,1.1].

[0090] S230: The local feature vector is updated to a global feature vector through a multi-head attention mechanism, and several of the global feature vectors constitute a global feature sequence;

[0091] Specifically, one of the local feature vectors is selected as the reference vector, and all of the local feature vectors are selected as reference vectors (including the reference vector). The reference query vector corresponding to the reference vector is obtained, and the reference key vector and reference value vector corresponding to the reference vector are obtained. The attention score between the reference vector and the reference vector is obtained based on the reference query vector, the reference key vector, and the reference value vector. Several attention scores are normalized to form a fusion weight. A global feature vector corresponding to the reference vector is generated based on the fusion weight and the reference vector.

[0092] S240: Perform dimensionality equalization on all the global feature vectors to obtain an aggregate vector;

[0093] Taking 1024 dimensions as an example, each global feature vector has 1024-dimensional vector values. In each dimension, the vector values ​​of all global feature vectors in that dimension are averaged to obtain the dimension vector mean. All dimension vector mean values ​​are aggregated into the aggregate vector. The sequence length of the aggregate vector is 1, and the dimension is still 1024.

[0094] S250: Perform a linear transformation on the aggregated vector to convert the aggregated vector into a raw category score dataset, the raw category score dataset including several category scores, the category scores corresponding to preset disease categories;

[0095] The formula for obtaining the category score is:

[0096] ,

[0097] in, This represents the category score corresponding to the i-th preset disease category. Represents an aggregate vector. Denotes the weight matrix, and , Indicates the bias term, and ,in, This represents the dimension of the aggregate vector. This indicates the total number of preset disease categories, i.e. Characterizes the Dimension mapping to There are several categories; it is understood that multiple preset disease categories can be set in advance, and the multiple preset disease categories can be transverse cracks, longitudinal cracks, network cracks, block cracks, potholes, ruts, etc. It should be noted that one of the preset disease categories is no disease, that is, there is no disease.

[0098] S260: Based on the category score, obtain the disease probability corresponding to the preset disease category, and select the preset disease category with the highest disease probability as the disease category to be measured;

[0099] Understandably, a total score is obtained based on all the scores of the categories, and the score of each category is divided by the total score to obtain the disease probability.

[0100] By adding the position encoding, each global feature vector in the global feature sequence corresponds to a grid region, ensuring that position information is preserved during CNN grid processing and improving the accuracy of crack direction judgment. By introducing a multi-head attention mechanism, the local feature vectors can interact directly, and the crack initiation feature can focus on the initiation feature, thereby connecting fragmented cracks into a complete form and improving the recognition accuracy of network cracks and discontinuous cracks.

[0101] S30: Perform edge detection on the enhanced road surface image to obtain a contour image; obtain a semantic feature tensor and a contour feature tensor based on the enhanced road surface image and the contour image respectively; obtain a binarized segmentation map through the semantic feature tensor and the contour feature tensor; the binarized segmentation map includes the diseased area.

[0102] Edge detection algorithms are widely used and will not be elaborated upon here. After acquiring the contour image, the enhanced road surface image and the contour image are input into two independent encoder paths for feature extraction and downsampling. The essence is the same as the principle of obtaining the texture feature map, and will not be elaborated upon here. The enhanced road surface image is processed through a series of convolutional and max-pooling layers. At the last layer of the encoding path, the semantic feature tensor is obtained. The semantic feature tensor corresponds to the global context and semantic information of the image. The contour feature tensor is obtained in the same way, corresponding to the edge structure and position information of the image.

[0103] Step S30 includes:

[0104] S310: Concatenate the semantic feature tensor and the contour feature tensor into a combined tensor, and obtain a spatial attention map based on the combined tensor;

[0105] The formula for obtaining the spatial attention map is:

[0106] ,

[0107] in, Representing a spatial attention map, Represents a combinatorial tensor. This represents a 1x1 convolution operation. This represents the Sigmoid activation function;

[0108] S320: The semantic feature tensor is weighted and modulated using the spatial attention map to obtain an enhanced feature tensor, and the enhanced feature tensor is upsampled to obtain a single-channel feature map with the same size as the enhanced road surface image;

[0109] The formula for obtaining the enhanced feature tensor is:

[0110] ,

[0111] in, Represents the augmented feature tensor. Representing a spatial attention map, This indicates a residual join operation. It represents the Hadamah accumulation. Represents a semantic feature tensor;

[0112] After obtaining the enhanced feature tensor, the enhanced feature tensor is input into the decoder and upsampled through a series of upsampling layers and convolutional layers to obtain the single-channel feature map.

[0113] S330: Map the original pixel value of each pixel in the single-channel feature map to a pixel probability, compare the pixel probability with a probability threshold, and determine the final pixel value of the pixel. The final pixel value is 0 or 1, so as to convert the single-channel feature map into a binary segmentation map.

[0114] The formula for obtaining the pixel probability is:

[0115] ,

[0116] in, This represents the pixel probability of the pixel at position (i, j) in the single-channel feature map. This represents the original pixel value of the pixel at position (i, j) in the single-channel feature map. Represents an exponential function;

[0117] After obtaining the pixel probability, if the pixel probability is greater than or equal to a probability threshold, the final pixel value of the pixel corresponding to that pixel probability is selected as 1; if the pixel probability is less than the probability threshold, the final pixel value of the pixel corresponding to that pixel probability is selected as 0. Understandably, the area with a final pixel value of 1 is the diseased area, and the area with a final pixel value of 0 is the background area.

[0118] Compared to directly obtaining the disease area through the contour image, fusing the enhanced pavement image and the contour image into the binary segmentation map not only solves the problem of insufficient single-modal information, but also combines the precise boundary positioning capability of the contour image with the rich semantic discrimination capability of the enhanced pavement image to provide semantic annotation for the identified contour, effectively improving the accuracy of disease area identification.

[0119] S40: Obtain a road surface score based on the determined disease category and the binarized segmentation map, and compare the road surface score with a score threshold to determine whether road maintenance is required;

[0120] Step S40 includes:

[0121] S410: Obtain the disease weight corresponding to the measured disease category, and obtain the disease area of ​​the disease region;

[0122] The disease weights can be set based on experience, according to different types.

[0123] S420: Obtain a pavement score based on the defect weight and the defect area;

[0124] The formula for obtaining the road surface score is:

[0125] ,

[0126] in, Indicates road surface score. This represents the disease weight corresponding to the d-th measured disease category. This represents the disease area corresponding to the d-th measured disease category. Indicates the total road surface area. This indicates the total number of disease categories measured. , All of these represent empirical coefficients; it should be noted that the total number of disease categories determined does not include those without disease, and for any road, the sum of the areas of diseased regions of all the initial road surface images corresponding to the road is obtained to form the diseased area.

[0127] After obtaining the road surface score, if the road surface score is greater than the score threshold, it is determined that road maintenance is required; if the road surface score is less than the score threshold, it is determined that road maintenance is not required.

[0128] By acquiring the enhanced illumination component and the enhanced reflection component, an enhanced pavement image with uniform illumination, prominent details, and controlled noise is obtained, providing a stable and highly separable input for subsequent processing, laying the foundation for detecting early-stage, low-contrast defects. By acquiring the global feature sequence, the defect category is determined based on the understanding of the overall morphology and spatial layout of the defects, improving the accuracy of identifying defect categories with complex morphology and discontinuous structure. Combined with the enhanced pavement image, accurate detection of early-stage defects is achieved. By combining the semantic feature tensor and the contour feature tensor, false detections caused by non-defect edges such as pavement markings, gravel textures, and water stains are effectively suppressed, significantly improving the signal-to-noise ratio of the segmentation results and further enhancing the detection accuracy of defects. By acquiring the pavement score, a complete closed loop for road maintenance judgment is achieved, transforming post-maintenance into predictive maintenance.

[0129] Please see Figure 2 The second embodiment of the present invention provides a road maintenance discrimination system, which is applied to the road maintenance discrimination method described in the above embodiments, and will not be repeated hereafter. As used below, the terms "module," "unit," "subunit," etc., can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0130] The system includes:

[0131] The acquisition module 10 is used to acquire an initial road surface image, convert the initial road surface image into an enhanced illumination component and an enhanced reflection component, and acquire an enhanced road surface image through the enhanced illumination component and the enhanced reflection component;

[0132] The acquisition module 10 includes:

[0133] The first unit is used to construct an initial optimization model and obtain a training input set, which includes training road surface images. The training input set is used as the input value of the initial optimization model, and the initial optimization model outputs training illumination output components and training reflection output components.

[0134] The second unit is used to construct a reconstruction loss function, a reflection component constraint loss function, and an illumination component constraint loss function based on the training illumination output component and the training reflection output component, respectively, and to construct a total loss function through the reconstruction loss function, the reflection component constraint loss function, and the illumination component constraint loss function.

[0135] The third unit is used to update the initial optimization model to the final optimization model through the total loss function, and to use the initial road surface image as the input value of the final optimization model, so as to obtain the enhanced illumination component and the enhanced reflection component through the final optimization model;

[0136] Analysis module 20 is used to acquire a texture feature map corresponding to the enhanced pavement image, convert the texture feature map into a global feature sequence, and acquire the measured disease category corresponding to the global feature sequence;

[0137] The analysis module 20 includes:

[0138] The fourth unit is used to extract the texture feature map of the enhanced road surface image through a CNN network, wherein the texture feature map includes several grid regions;

[0139] The fifth unit is used to flatten the texture feature map into a texture feature sequence in the spatial dimension. The texture feature sequence includes several texture feature vectors corresponding to the grid region. Position encoding is added to the texture feature vectors to obtain several local feature vectors.

[0140] The sixth unit is used to update the local feature vectors into global feature vectors through a multi-head attention mechanism, wherein several of the global feature vectors constitute a global feature sequence;

[0141] The seventh unit is used to perform dimensionality equalization processing on all the global feature vectors to obtain an aggregate vector;

[0142] The eighth unit is used to perform a linear transformation on the aggregated vector to convert the aggregated vector into a raw category score dataset, which includes several category scores and the category scores correspond to preset disease categories;

[0143] The ninth unit is used to obtain the disease probability corresponding to the preset disease category based on the category score, and select the preset disease category with the highest disease probability as the disease category to be measured.

[0144] Extraction module 30 is used to perform edge detection on the enhanced road surface image to obtain a contour image, obtain a semantic feature tensor and a contour feature tensor based on the enhanced road surface image and the contour image respectively, and obtain a binarized segmentation map through the semantic feature tensor and the contour feature tensor, wherein the binarized segmentation map includes the disease area.

[0145] The extraction module 30 includes:

[0146] The tenth unit is used to concatenate the semantic feature tensor and the contour feature tensor into a combined tensor, and to obtain a spatial attention map based on the combined tensor;

[0147] The eleventh unit is used to perform weighted modulation on the semantic feature tensor through the spatial attention map to obtain an enhanced feature tensor, and to upsample the enhanced feature tensor to obtain a single-channel feature map with the same size as the enhanced road surface image.

[0148] The twelfth unit is used to map the original pixel value of each pixel in the single-channel feature map to a pixel probability, compare the pixel probability with a probability threshold, and determine the final pixel value of the pixel, wherein the final pixel value is 0 or 1, so as to convert the single-channel feature map into a binary segmentation map.

[0149] The execution module 40 is used to obtain a road surface score based on the determined disease category and the binarized segmentation map, and compare the road surface score with a score threshold to determine whether road maintenance is required.

[0150] The execution module 40 includes:

[0151] The thirteenth unit is used to obtain the disease weight corresponding to the measured disease category and to obtain the disease area of ​​the disease region;

[0152] The fourteenth unit is used to obtain a pavement score based on the defect weight and the defect area.

[0153] The present invention also provides a computer, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the road maintenance determination method as described in the above technical solutions.

[0154] The present invention also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the road maintenance determination method as described in the above technical solution.

[0155] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0156] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for determining road maintenance, characterized in that, Includes the following steps: An initial road surface image is acquired, and the initial road surface image is converted into an enhanced illumination component and an enhanced reflection component. An enhanced road surface image is then obtained through the enhanced illumination component and the enhanced reflection component. The step of converting the initial road surface image into an enhanced illumination component and an enhanced reflection component includes: An initial optimization model is constructed, and a training input set is obtained, which includes training road surface images. The training input set is used as the input value of the initial optimization model, and the initial optimization model outputs training illumination output components and training reflection output components. Based on the training illumination output component and the training reflection output component, a reconstruction loss function, a reflection component constraint loss function, and an illumination component constraint loss function are constructed respectively, and a total loss function is constructed through the reconstruction loss function, the reflection component constraint loss function, and the illumination component constraint loss function. The expression for the reconstruction loss function is: ， in, Represents the reconstruction loss function. Represents the training road surface image, This represents the training reflection output component. This represents the training illumination output component. This indicates element-wise multiplication. Represents the L1 norm; The expression for the reflection component constraint loss function is: ， in, Represents the reflection component constraint loss function. The V channel represents the training reflection output component. This represents the V channel of the training road surface image. Represents the HSV color space. Indicates channel extraction operation. This indicates an adaptive histogram equalization operation. Indicates the noise suppression coefficient. This represents the image gradient of the training reflection output component; The expression for the illumination component constraint loss function is: ， in, Represents the illumination component constraint loss function. This represents the image gradient of the training illumination output component. Represents the coupling coefficient. Represents an exponential function; The initial optimization model is updated to the final optimization model through the total loss function, and the initial road surface image is used as the input value of the final optimization model to obtain the enhanced illumination component and the enhanced reflection component through the final optimization model. Obtain the texture feature map corresponding to the enhanced road surface image, convert the texture feature map into a global feature sequence, and obtain the determination disease category corresponding to the global feature sequence; Edge detection is performed on the enhanced road surface image to obtain a contour image. Semantic feature tensors and contour feature tensors are obtained based on the enhanced road surface image and the contour image, respectively. A binarized segmentation map is obtained through the semantic feature tensor and the contour feature tensor, and the binarized segmentation map includes the diseased area. Based on the determined disease category and the binarized segmentation map, a road surface score is obtained. The road surface score is then compared with a score threshold to determine whether road maintenance is required.

2. The road maintenance determination method according to claim 1, characterized in that, The step of obtaining the texture feature map corresponding to the enhanced road surface image and converting the texture feature map into a global feature sequence includes: The texture feature map of the enhanced road surface image is extracted using a CNN network, and the texture feature map includes several grid regions; In the spatial dimension, the texture feature map is flattened into a texture feature sequence, which includes several texture feature vectors corresponding to the grid region. Position encoding is added to the texture feature vectors to obtain several local feature vectors. The local feature vectors are updated to global feature vectors through a multi-head attention mechanism, and several of the global feature vectors constitute a global feature sequence.

3. The road maintenance determination method according to claim 2, characterized in that, The formula for obtaining the location code is: ， in, This represents the value of the position code in the 2i-th dimension. This represents the position code value in the (2i+1)th dimension. This represents the position index corresponding to the grid region, and 0 ≤ < ,in Indicates the total number of grid regions. Represents a dimension index, and 0 ≤ < , This represents the total dimension of the texture feature vector.

4. The road maintenance determination method according to claim 2, characterized in that, The step of obtaining the disease category corresponding to the global feature sequence includes: Dimensional equalization is performed on all the global feature vectors to obtain aggregate vectors; The aggregated vector is linearly transformed to convert it into a raw category score dataset, which includes several category scores that correspond to preset disease categories. Based on the category score, the disease probability corresponding to the preset disease category is obtained, and the preset disease category with the highest disease probability is selected as the disease category to be measured.

5. The road maintenance determination method according to claim 4, characterized in that, The formula for obtaining the category score is: ， in, This represents the category score corresponding to the i-th preset disease category. Represents an aggregate vector. Represents the weight matrix. This indicates the bias term.

6. The road maintenance determination method according to claim 1, characterized in that, The step of obtaining the binarized segmentation map using the semantic feature tensor and the contour feature tensor includes: The semantic feature tensor and the contour feature tensor are concatenated into a combined tensor, and a spatial attention map is obtained based on the combined tensor. The semantic feature tensor is weighted and modulated using the spatial attention map to obtain an enhanced feature tensor. The enhanced feature tensor is then upsampled to obtain a single-channel feature map with the same size as the enhanced road surface image. The original pixel value of each pixel in the single-channel feature map is mapped to a pixel probability. The pixel probability is compared with a probability threshold to determine the final pixel value of the pixel. The final pixel value is 0 or 1, so as to convert the single-channel feature map into a binary segmentation map.

7. The road maintenance determination method according to claim 1, characterized in that, The step of obtaining a pavement score based on the determined disease category and the binarized segmentation map includes: Obtain the disease weight corresponding to the measured disease category, and obtain the disease area of ​​the disease region; The pavement score is obtained based on the defect weight and the defect area.

8. A road maintenance discrimination system, applied to the road maintenance discrimination method as described in any one of claims 1 to 7, characterized in that, The system includes: The acquisition module is used to acquire an initial road surface image, convert the initial road surface image into an enhanced illumination component and an enhanced reflection component, and acquire an enhanced road surface image through the enhanced illumination component and the enhanced reflection component. The analysis module is used to obtain a texture feature map corresponding to the enhanced pavement image, convert the texture feature map into a global feature sequence, and obtain the measured disease category corresponding to the global feature sequence; An extraction module is used to perform edge detection on the enhanced road surface image to obtain a contour image, obtain a semantic feature tensor and a contour feature tensor based on the enhanced road surface image and the contour image respectively, and obtain a binarized segmentation map through the semantic feature tensor and the contour feature tensor, wherein the binarized segmentation map includes the diseased area. The execution module is used to obtain a pavement score based on the determined disease category and the binarized segmentation map, and compare the pavement score with a score threshold to determine whether road maintenance is required.

Images