Road maintenance discrimination method and system

Abstract

The invention provides a road maintenance discrimination method and system, and the method comprises the steps: converting an initial road surface image into an enhanced illumination component and an enhanced reflection component, so as to obtain an enhanced road surface image; obtaining a texture feature map corresponding to the enhanced pavement image, further obtaining a global feature sequence, and obtaining a determined disease category corresponding to the global feature sequence; acquiring a contour image based on the enhanced road surface image, acquiring a semantic feature tensor and a contour feature tensor through the enhanced road surface image and the contour image, and further acquiring a binary segmentation image including a disease area; and obtaining a road surface score based on the determined disease category and the binarization segmentation map to judge whether to carry out road maintenance or not. Through obtaining a global feature sequence, the recognition accuracy of disease categories is improved, and accurate detection of early diseases is realized in combination with an enhanced pavement image. The combination of the semantic feature tensor and the contour feature tensor effectively inhibits false detection generated by non-disease edges, and further improves the disease detection precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a road maintenance discrimination method and system. BACKGROUND

[0002] The highway is the core infrastructure of the modern transportation system, and its quality directly affects the transportation efficiency and driving safety. As an important carrier of people's life, the highway drives the economic development of the whole city, and its importance in urban construction and development has been increasingly concerned by everyone.

[0003] With the increase of the service life of the highway, its quality will gradually decrease due to the influence of vehicle load, climate change and natural environment for a long time, and diseases such as cracks will occur. If not maintained in time, it will further expand and accelerate the aging of the highway, not only increase the maintenance cost, but also may lead to traffic accidents and threaten public safety. Therefore, it is of great significance to prevent the occurrence of highway safety accidents to find and repair the pavement diseases in time.

[0004] At present, the disease identification is generally completed by detecting the highway pavement diseases, and then the road maintenance is carried out according to the diseases. However, the process of disease identification generally adopts sensor detection, which relies on sensors such as accelerometers and strain gauges installed on the carrier to collect signals. Its coverage range is large, but the detection precision is insufficient, and it is difficult to identify early disease conditions. SUMMARY

[0005] In view of the defects of the prior art, the purpose of the present application is to provide a road maintenance discrimination method and system, which aims to solve the technical problems that the detection precision is insufficient and it is difficult to identify early diseases in the process of road maintenance discrimination in the prior art.

[0006] In order to achieve the above purpose, in a first aspect, the present application provides a road maintenance discrimination method, comprising the following steps: An initial pavement image is acquired, the initial pavement image is converted into an enhanced illumination component and an enhanced reflection component, and an enhanced pavement image is acquired through the enhanced illumination component and the enhanced reflection component; A texture feature map corresponding to the enhanced pavement image is acquired, the texture feature map is converted into a global feature sequence, and a measured disease category corresponding to the global feature sequence is acquired; Edge detection is performed on the enhanced pavement image to acquire a contour image, a semantic feature tensor and a contour feature tensor are respectively acquired based on the enhanced pavement image and the contour image, a binary segmentation map is acquired through the semantic feature tensor and the contour feature tensor, and the binary segmentation map includes a disease area; Based on the determined disease category and the binary segmentation map, a road surface score is obtained, and the road surface score is compared with a score threshold to determine whether to perform road maintenance.

[0007] Further, the step of converting the initial road surface image into an enhanced illumination component and an enhanced reflection component comprises: An initial optimization model is constructed, a training input set is obtained, the training input set including a training road surface image, the training input set being an input value of the initial optimization model, and a training illumination output component and a training reflection output component being output by the initial optimization model; Based on the training illumination output component and the training reflection output component, a reconstruction loss function, a reflection component constraint loss function, and an illumination component constraint loss function are constructed, and a total loss function is constructed by the reconstruction loss function, the reflection component constraint loss function, and the illumination component constraint loss function; The initial optimization model is updated into a final optimization model by the total loss function, the initial road surface image being an input value of the final optimization model, and an enhanced illumination component and an enhanced reflection component being obtained by the final optimization model.

[0008] Further, the expression of the reconstruction loss function is: , wherein, L represents the reconstruction loss function, I represents the training road surface image, R represents the training reflection output component, I represents the training illumination output component, * represents element-wise multiplication, ||·|| represents an L1 norm; The expression of the reflection component constraint loss function is: , wherein, L represents the reflection component constraint loss function, V represents a V channel of the training reflection output component, V represents a V channel of the training road surface image, HSV represents an HSV color space, Ch represents a channel extraction operation, HE represents an adaptive histogram equalization operation, σ represents a noise suppression coefficient, ∇I represents an image gradient of the training reflection output component; The expression of the illumination component constraint loss function is: , wherein, denotes an illumination component constraint loss function, denotes an image gradient of a training illumination output component, denotes a coupling coefficient, denotes an exponential function.

[0009] Further, the step of obtaining a texture feature map corresponding to the enhanced road surface image and converting the texture feature map into a global feature sequence comprises: extracting a texture feature map of the enhanced road surface image through a CNN network, the texture feature map comprising a plurality of grid regions; flattening the texture feature map into a texture feature sequence in a spatial dimension, the texture feature sequence comprising a plurality of texture feature vectors corresponding to the grid regions, and adding position encoding to the texture feature vectors to obtain a plurality of local feature vectors; updating the local feature vectors into global feature vectors through a multi-head attention mechanism, and the plurality of global feature vectors forming a global feature sequence.

[0010] Further, the formula for obtaining the position encoding is: , wherein, denotes a value of the position encoding in the 2i-th dimension, denotes a value of the position encoding in the 2i+1-th dimension, denotes a position index corresponding to a grid region, and 0≤ < wherein denotes a total number of grid regions, denotes a dimension index, and 0≤ < , denotes a total dimension of the texture feature vector.

[0011] Further, the step of obtaining a determined disease category corresponding to the global feature sequence comprises: performing dimension equalization processing on all the global feature vectors to obtain an aggregated vector; performing linear transformation on the aggregated vector to convert the aggregated vector into a category raw score data set, the category raw score data set comprising a plurality of category scores, the category scores corresponding to preset disease categories; obtaining a disease probability corresponding to the preset disease category based on the category scores, and selecting a preset disease category with the maximum disease probability as the determined disease category.

[0012] Further, the formula for obtaining the category score is: , wherein, represents a class score corresponding to the i-th preset disease class, represents an aggregation vector, represents a weight matrix, represents a bias term.

[0013] Further, the step of obtaining a binary segmentation map through the semantic feature tensor and the contour feature tensor comprises: splicing the semantic feature tensor and the contour feature tensor into a combined tensor, and obtaining a spatial attention map based on the combined tensor; weighting and modulating the semantic feature tensor through the spatial attention map to obtain an enhanced feature tensor, and upsampling the enhanced feature tensor to obtain a single-channel feature map with the same size as the enhanced road image; mapping the original pixel value of each pixel point in the single-channel feature map to a pixel probability, comparing the pixel probability with a probability threshold, determining the final pixel value of the pixel point, and the final pixel value is 0 or 1, so as to convert the single-channel feature map into a binary segmentation map.

[0014] Further, the step of obtaining a road surface score based on the determined disease class and the binary segmentation map comprises: obtaining a disease weight corresponding to the determined disease class, and obtaining a disease area of the disease area; obtaining a road surface score based on the disease weight and the disease area.

[0015] In a second aspect, the embodiments of the present application provide a road maintenance discrimination system, which is applied to the road maintenance discrimination method as described in the first aspect above, and the system comprises: an obtaining module, configured to obtain an initial road image, convert the initial road image into an enhanced illumination component and an enhanced reflection component, and obtain an enhanced road image through the enhanced illumination component and the enhanced reflection component; an analysis module, configured to obtain a texture feature map corresponding to the enhanced road image, convert the texture feature map into a global feature sequence, and obtain a determined disease class corresponding to the global feature sequence; an extraction module, configured to perform edge detection on the enhanced road image to obtain a contour image, obtain a semantic feature tensor and a contour feature tensor based on the enhanced road image and the contour image respectively, and obtain a binary segmentation map through the semantic feature tensor and the contour feature tensor, wherein the binary segmentation map comprises a disease area; The execution module is configured to acquire a road surface score based on the determined disease category and the binary segmentation image, compare the road surface score with a score threshold, and determine whether to perform road maintenance.

[0016] In a third aspect, an embodiment of the present application provides a computer, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the road maintenance determination method according to the first aspect when executing the computer program.

[0017] In a fourth aspect, an embodiment of the present application provides a storage medium having a computer program stored thereon, and the computer program is executable by a processor to implement the road maintenance determination method according to the first aspect.

[0018] Compared with the prior art, the present application has the beneficial effects that: by acquiring the enhanced illumination component and the enhanced reflection component, and then acquiring the enhanced road surface image with uniform illumination, prominent details, and controlled noise, a stable quality and high feature separability input is provided for subsequent processing, and a foundation is laid for detecting early and low-contrast diseases; by acquiring the global feature sequence, the acquisition of the determined disease category is completed based on the understanding of the overall morphology and spatial layout of the disease, the recognition accuracy of the disease category with complex morphology and discontinuous structure is improved, and the early disease is accurately detected in combination with the enhanced road surface image; by combining the semantic feature tensor and the contour feature tensor, the false detection caused by the non-disease edges such as road markings, stone textures, and water stains is effectively suppressed, the signal-to-noise ratio of the segmentation result is significantly improved, and the detection accuracy of the disease is further improved; by acquiring the road surface score, the overall closed loop of road maintenance determination is completed, and the post-maintenance is converted into predictive maintenance. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 The flowchart of the road maintenance determination method in the first embodiment of the present application; Figure 2 The structural block diagram of the road maintenance determination system in the second embodiment of the present application; The following specific embodiments will further illustrate the present application in combination with the above drawings. DETAILED DESCRIPTION

[0020] In order to facilitate the understanding of the present application, the present application will be described more fully below with reference to the related drawings. The drawings show several embodiments of the present application. However, the present application can be implemented in many different forms, and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0021] It should be understood that when an element as a layer, region, or substrate is referred to as being "on" another element, it can be directly on the other element or intervening elements can also be present. In addition, it should be understood that when an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements can also be present. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0023] Referring to Figure 1 The road maintenance discrimination method provided by the first embodiment of the application comprises the following steps: S10: An initial road surface image is acquired, the initial road surface image is converted into an enhanced illumination component and an enhanced reflection component, and an enhanced road surface image is acquired through the enhanced illumination component and the enhanced reflection component; In this embodiment, a photograph is acquired in the manner of aerial photography by a UAV, and the photograph is grid-divided to form a plurality of initial road surface images of smaller sizes, so as to avoid the case that multiple diseases exist in the initial road surface image. The initial road surface image is affected by the shooting environment (outdoor illumination is affected by weather, time, and obstructions), so that the gray difference between the diseases existing on the road surface and the road surface in the dark area or the bright area becomes extremely low, the diseases are hidden in the background, and cannot be effectively detected; and the signal strength of early diseases is far lower than that of normal road surface textures and noises, the signal-to-noise ratio of the characteristics of early diseases in the initial road surface image is too low, and the early diseases are difficult to be identified. Through acquisition of the enhanced road surface image, accurate identification of diseases in the subsequent process can be facilitated.

[0024] The step S10 comprises: S110: An initial optimization model is constructed, a training input set is acquired, the training input set comprises a training road surface image, the training input set is taken as an input value of the initial optimization model, and a training illumination output component and a training reflection output component are output through the initial optimization model; S120: A reconstruction loss function, a reflection component constraint loss function, and an illumination component constraint loss function are respectively constructed based on the training illumination output component and the training reflection output component, and a total loss function is constructed through the reconstruction loss function, the reflection component constraint loss function, and the illumination component constraint loss function; In the embodiment, the initial optimization model is an unsupervised deep learning network model. Since absolute correct illumination truth value and reflection truth value corresponding to the training road surface image cannot be obtained, the reconstruction loss function, the reflection component constraint loss function and the illumination component constraint loss function are set to measure the accuracy of the training illumination output component and the training reflection output component, and the reconstruction loss function, the reflection component constraint loss function and the illumination component constraint loss function correspond to physical rules, expected attributes of a reflection map and expected attributes of an illumination map respectively.

[0025] The expression of the reconstruction loss function is as follows: , Among them, the reconstruction loss function is denoted by Lr, the training road surface image is denoted by I, the training reflection output component is denoted by R, the training illumination output component is denoted by I, element-wise multiplication is denoted by, L1 norm is denoted by ||·||1; the reconstruction loss function makes the training illumination output component and the training reflection output component inverse operations of each other, ensures that the aggregation is close to the training road surface image, and ensures the basic correctness of the decomposition process.

[0026] The expression of the reflection component constraint loss function is as follows: , Among them, the reflection component constraint loss function is denoted by Lr, the V channel of the training reflection output component is denoted by Rv, the V channel of the training road surface image is denoted by Iv, HSV color space is denoted by HSV, channel extraction operation is denoted by, adaptive histogram equalization operation is denoted by, noise suppression coefficient is denoted by k, image gradient of the training reflection output component is denoted by ||R||x; by introducing the adaptive histogram equalization operation to process the V channel of the training road surface image, the V channel is used as a reference template for detail enhancement, and then the gradient of the reflection map is gradually reduced, which embodies the prior condition that the reflection map should suppress noise and unnecessary texture mutation.

[0027] The expression of the illumination component constraint loss function is as follows: , Among them, the illumination component constraint loss function is denoted by Li, image gradient of the training illumination output component is denoted by ||I||x, Represents the coupling coefficient. The exponential function represents the lighting component constraint loss function, which allows abrupt changes in lighting at object edges, while forcing smooth lighting in flat areas, thereby ensuring that the trained lighting output components conform to the physical condition that lighting is piecewise smooth.

[0028] The expression for the total loss function is: , in, Represents the total loss function. , All of these represent hyperparameters. In this embodiment, The value is 0.5. The value is 0.1; S130: The initial optimization model is updated to the final optimization model through the total loss function, and the initial road surface image is used as the input value of the final optimization model to obtain the enhanced illumination component and the enhanced reflection component through the final optimization model; After obtaining the enhanced illumination component and the enhanced reflection component, the enhanced road surface image is obtained by multiplying them element-wise. The enhanced road surface image has the characteristics of balanced brightness and prominent details. The entire optimization process of the final optimization model only requires the input of the training road surface image, without any "standard enhancement result" as a label. Its supervision signal comes from the total loss function formed by the reconstruction loss function, the reflection component constraint loss function, and the illumination component constraint loss function, which reduces the data threshold and cost of model training. It achieves targeted, adaptive, and high-fidelity enhancement of the details of defects in the image, while effectively suppressing noise and illumination interference, providing an accurate basis for subsequent defect identification.

[0029] S20: Obtain the texture feature map corresponding to the enhanced road surface image, convert the texture feature map into a global feature sequence, and obtain the disease category corresponding to the global feature sequence; Step S20 includes: S210: Extract the texture feature map of the enhanced road surface image through a CNN network, wherein the texture feature map includes several grid regions; Specifically, the enhanced road surface image (such as 512*512 pixels) is input into the CNN network, first, a 7*7 convolution kernel is used for preliminary feature extraction to obtain a plurality of output feature maps, the output feature map corresponds to a local region centered on a pixel point in the enhanced road surface image, the size of the local region is equal to the size of the convolution kernel (7*7), then a maximum pooling is used to reduce the resolution of the output feature map to retain significant features, such as a 2*2 pooling window reduces the size of the output feature map by half to reduce the amount of calculation, while enhancing the translation invariance of the texture, a plurality of output feature maps after the pooling are synchronously input into a series of residual blocks for deep processing, the shallow residual block learns to capture simple local texture patterns (such as the edges of cracks, line segments), and the deep residual block learns to capture more complex and global texture patterns (such as the staggered shape of cracks), and then generates the texture feature map (for example, the sampling rate is 32 times, the structure of the texture feature map is 16*16*1024), based on this, the texture feature map has 16*16=256 grid regions, and the feature dimension of each grid region is 1024 dimensions. The CNN network has been widely used, and thus will not be described here.

[0030] S220: flattening the texture feature map into a texture feature sequence in the spatial dimension, the texture feature sequence including a plurality of texture feature vectors corresponding to the grid regions, adding position encoding to the texture feature vectors to obtain a plurality of local feature vectors; Specifically, a position index is generated based on the position relationship of the grid regions in the texture feature map, assuming that the texture feature map includes 3*3=9 grids, and the feature dimension of each grid is 4, for grid (1, 1), the position index is 0, based on the position index, the position encoding of 4 dimensions needs to be generated.

[0031] The formula for obtaining the position encoding is: , wherein, represents the value of the position encoding in the 2i-th dimension, represents the value of the position encoding in the 2i+1-th dimension, represents the position index corresponding to the grid region, and 0≤ < wherein represents the total number of grid regions, represents the dimension index, and 0≤ < , represents the total dimension of the texture feature vector.

[0032] ​When i takes the value of 0, PE(0, 0) = 0, PE(0, 1) = 1, when i takes the value of 1, PE(0, 2) = 0, PE(0, 3) = 1, then the position encoding is [0, 1, 0, 1], assuming that the texture feature vector corresponding to the grid area is [0.5, 0.2, 0.8, 0.1], then the two are summed channel by channel to generate a local feature vector [0.5, 1.2, 0.8, 1.1].

[0033] S230: updating the local feature vectors to global feature vectors through a multi-head attention mechanism, and a plurality of global feature vectors constitute a global feature sequence; Specifically, one of the local feature vectors is selected as a reference vector, and all of the local feature vectors are selected as reference vectors (including the reference vector), a reference query vector corresponding to the reference vector is obtained, and a reference key vector and a reference value vector corresponding to the reference vector are obtained, an attention score between the reference vector and the reference vector is obtained based on the reference query vector, the reference key vector and the reference value vector, a plurality of attention scores are normalized to form a fusion weight, and a global feature vector corresponding to the reference vector is generated based on the fusion weight and the reference vector.

[0034] S240: performing dimension equalization processing on all global feature vectors to obtain an aggregated vector; Taking 1024 dimensions as an example, each global feature vector has a 1024-dimensional vector value, and in each dimension, the vector values of all global feature vectors in the dimension are processed by mean value to obtain a dimension vector mean value, and all dimension vector mean values are aggregated into the aggregated vector, the sequence length of the aggregated vector is 1, and the dimension is still 1024.

[0035] S250: performing linear change on the aggregated vector to convert the aggregated vector into a category raw score data set, the category raw score data set including a plurality of category scores, and the category scores corresponding to preset disease categories; The formula for obtaining the category score is: , Wherein, represents the category score corresponding to the i-th preset disease category, represents the aggregated vector, represents a weight matrix, and , represents a bias term, and , wherein, represents the dimension of the aggregated vector, represents the total number of preset disease categories, that is, Characterizes the Dimension mapping to There are several categories; it is understood that multiple preset disease categories can be set in advance, and the multiple preset disease categories can be transverse cracks, longitudinal cracks, network cracks, block cracks, potholes, ruts, etc. It should be noted that one of the preset disease categories is no disease, that is, there is no disease.

[0036] S260: Based on the category score, obtain the disease probability corresponding to the preset disease category, and select the preset disease category with the highest disease probability as the disease category to be measured; Understandably, a total score is obtained based on all the scores of the categories, and the score of each category is divided by the total score to obtain the disease probability.

[0037] By adding the position encoding, each global feature vector in the global feature sequence corresponds to a grid region, ensuring that position information is preserved during CNN grid processing and improving the accuracy of crack direction judgment. By introducing a multi-head attention mechanism, the local feature vectors can interact directly, and the crack initiation feature can focus on the initiation feature, thereby connecting fragmented cracks into a complete form and improving the recognition accuracy of network cracks and discontinuous cracks.

[0038] S30: Perform edge detection on the enhanced road surface image to obtain a contour image; obtain a semantic feature tensor and a contour feature tensor based on the enhanced road surface image and the contour image respectively; obtain a binarized segmentation map through the semantic feature tensor and the contour feature tensor; the binarized segmentation map includes the diseased area. Edge detection algorithms are widely used and will not be elaborated upon here. After acquiring the contour image, the enhanced road surface image and the contour image are input into two independent encoder paths for feature extraction and downsampling. The essence is the same as the principle of obtaining the texture feature map, and will not be elaborated upon here. The enhanced road surface image is processed through a series of convolutional and max-pooling layers. At the last layer of the encoding path, the semantic feature tensor is obtained. The semantic feature tensor corresponds to the global context and semantic information of the image. The contour feature tensor is obtained in the same way, corresponding to the edge structure and position information of the image.

[0039] Step S30 includes: S310: Concatenate the semantic feature tensor and the contour feature tensor into a combined tensor, and obtain a spatial attention map based on the combined tensor; The formula for obtaining the spatial attention map is: , in, a spatial attention map, a combined tensor, a 1x1 convolution operation, a sigmoid activation function; S320: weighting and modulating the semantic feature tensor through the spatial attention map to obtain an enhanced feature tensor, and upsampling the enhanced feature tensor to obtain a single-channel feature map with the same size as the enhanced road surface image; The formula for obtaining the enhanced feature tensor is: , wherein, the enhanced feature tensor, the spatial attention map, a residual connection operation, a Hadamard product, the semantic feature tensor; After obtaining the enhanced feature tensor, the enhanced feature tensor is input into a decoder, and is upscaled via a series of upsampling layers and convolution layers to obtain the single-channel feature map.

[0040] S330: mapping the original pixel value of each pixel point in the single-channel feature map to a pixel probability, comparing the pixel probability with a probability threshold, and determining the final pixel value of the pixel point, wherein the final pixel value is 0 or 1, so as to convert the single-channel feature map into a binary segmentation map. The formula for obtaining the pixel probability is: , wherein, the pixel probability of the pixel point at position (i, j) in the single-channel feature map, the original pixel value of the pixel point at position (i, j) in the single-channel feature map, an exponential function; After obtaining the pixel probability, if the pixel probability is greater than or equal to a probability threshold, the final pixel value of the pixel point corresponding to the pixel probability is selected as 1, and if the pixel probability is less than the probability threshold, the final pixel value of the pixel point corresponding to the pixel probability is selected as 0. It can be understood that the region with the final pixel value of 1 is a disease region, and the region with the final pixel value of 0 is a background region.

[0041] Compared with directly obtaining the disease area through the contour image, by fusing the enhanced road surface image and the contour image into the binary segmentation image, not only the problem of insufficient single modal information is solved, but also the precise boundary positioning capability of the contour image and the rich semantic discrimination capability of the enhanced road surface image are combined, semantic annotation is performed on the recognized contour, and the accuracy of disease area recognition is effectively improved.

[0042] S40: obtaining a road surface score based on the determined disease categories and the binary segmentation image, comparing the road surface score with a score threshold to determine whether to perform road maintenance; The step S40 includes: S410: obtaining a disease weight corresponding to the determined disease categories and obtaining a disease area of the disease area; The disease weight can be set according to different categories based on experience.

[0043] S420: obtaining a road surface score based on the disease weight and the disease area; The formula for obtaining the road surface score is: , wherein, represents the road surface score, represents a disease weight corresponding to the dth determined disease category, represents a disease area corresponding to the dth determined disease category, represents a total area of a road surface, represents a total number of determined disease categories, , both represent experience coefficients; it should be noted that the total number of determined disease categories does not include no disease, and the area sum of the disease area of all the initial road surface images corresponding to a road is obtained to form the disease area.

[0044] After obtaining the road surface score, if the road surface score is greater than a score threshold, it is determined that road maintenance is needed, and if the road surface score is less than the score threshold, it is determined that road maintenance is not needed.

[0045] By acquiring the enhanced illumination component and the enhanced reflection component, the enhanced road surface image with uniform illumination, prominent details and controlled noise is acquired, which provides a high-quality input with high feature separability for subsequent processing and lays a foundation for early detection of low-contrast diseases; by acquiring the global feature sequence, the acquisition of the disease category is completed based on the understanding of the overall morphology and spatial layout of the disease, which improves the recognition accuracy of the disease category with complex morphology and discontinuous structure, and realizes accurate detection of early diseases in combination with the enhanced road surface image; by combining the semantic feature tensor and the contour feature tensor, false detection caused by non-disease edges such as road markings, stone textures and water stains is effectively suppressed, the signal-to-noise ratio of the segmentation result is significantly improved, and the detection accuracy of the disease is further improved; by acquiring the road surface score, the overall closed loop of road maintenance discrimination is completed, and the post-maintenance is converted into predictive maintenance.

[0046] Referring to Figure 2 The second embodiment of the present application provides a road maintenance discrimination system, which is applied to the road maintenance discrimination method in the above-mentioned embodiments and has been described above. As used below, the terms "module", "unit", "sub-unit" and the like can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, hardware or a combination of software and hardware is also possible and contemplated.

[0047] The system comprises: The acquisition module 10 is configured to acquire an initial road surface image, convert the initial road surface image into an enhanced illumination component and an enhanced reflection component, and acquire an enhanced road surface image through the enhanced illumination component and the enhanced reflection component. The acquisition module 10 comprises: The first unit is configured to construct an initial optimization model, acquire a training input set, the training input set comprising a training road surface image, take the training input set as an input value of the initial optimization model, and output a training illumination output component and a training reflection output component through the initial optimization model. The second unit is configured to construct a reconstruction loss function, a reflection component constraint loss function and an illumination component constraint loss function based on the training illumination output component and the training reflection output component respectively, and construct a total loss function through the reconstruction loss function, the reflection component constraint loss function and the illumination component constraint loss function. The third unit is configured to update the initial optimization model to a final optimization model through the total loss function, take the initial road surface image as an input value of the final optimization model, and acquire an enhanced illumination component and an enhanced reflection component through the final optimization model. The analysis module 20 is configured to obtain a texture feature map corresponding to the enhanced road surface image, convert the texture feature map into a global feature sequence, and obtain a determined disease category corresponding to the global feature sequence. The analysis module 20 comprises: A fourth unit is configured to extract a texture feature map of the enhanced road surface image through a CNN network, and the texture feature map comprises a plurality of grid regions. A fifth unit is configured to flatten the texture feature map into a texture feature sequence in a spatial dimension, and the texture feature sequence comprises a plurality of texture feature vectors corresponding to the grid regions, and position encoding is added to the texture feature vectors to obtain a plurality of local feature vectors. A sixth unit is configured to update the local feature vectors into global feature vectors through a multi-head attention mechanism, and a plurality of the global feature vectors constitute a global feature sequence. A seventh unit is configured to perform dimension equalization processing on all the global feature vectors to obtain an aggregated vector. An eighth unit is configured to perform linear transformation on the aggregated vector to convert the aggregated vector into a category raw score data set, and the category raw score data set comprises a plurality of category scores corresponding to preset disease categories. A ninth unit is configured to obtain a disease probability corresponding to the preset disease category based on the category scores, and select a preset disease category with the maximum disease probability as the determined disease category. The extraction module 30 is configured to perform edge detection on the enhanced road surface image to obtain a contour image, obtain a semantic feature tensor and a contour feature tensor based on the enhanced road surface image and the contour image respectively, and obtain a binary segmentation map through the semantic feature tensor and the contour feature tensor, and the binary segmentation map comprises a disease area. The extraction module 30 comprises: A tenth unit is configured to splice the semantic feature tensor and the contour feature tensor into a combined tensor, and obtain a spatial attention map based on the combined tensor. An eleventh unit is configured to perform weighted modulation on the semantic feature tensor through the spatial attention map to obtain an enhanced feature tensor, and perform up-sampling on the enhanced feature tensor to obtain a single-channel feature map with the same size as the enhanced road surface image. A twelfth unit is configured to map an original pixel value of each pixel point in the single-channel feature map to a pixel probability, compare the pixel probability with a probability threshold, determine a final pixel value of the pixel point, and the final pixel value is 0 or 1, so as to convert the single-channel feature map into a binary segmentation map. The execution module 40 is configured to acquire a road surface score based on the determined disease category and the binary segmentation image, and compare the road surface score with a score threshold to determine whether to perform road maintenance. The execution module 40 includes: A thirteenth unit is configured to acquire a disease weight corresponding to the determined disease category, and acquire a disease area of the disease region. A fourteenth unit is configured to acquire a road surface score based on the disease weight and the disease area.

[0048] The present application also provides a computer including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the road maintenance determination method as described in the above technical solution.

[0049] The present application also provides a storage medium having a computer program stored thereon, and the computer program is executable on a processor to implement the road maintenance determination method as described in the above technical solution.

[0050] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0051] The above embodiments only express several embodiments of the present application, and the description is more specific and detailed, but it cannot be understood as a limitation on the scope of the present application. It should be noted that for ordinary skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which are all within the scope of protection of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A method for determining road maintenance, characterized in that, Includes the following steps: An initial road surface image is acquired, and the initial road surface image is converted into an enhanced illumination component and an enhanced reflection component. An enhanced road surface image is then obtained through the enhanced illumination component and the enhanced reflection component. Obtain the texture feature map corresponding to the enhanced road surface image, convert the texture feature map into a global feature sequence, and obtain the determination disease category corresponding to the global feature sequence; Edge detection is performed on the enhanced road surface image to obtain a contour image. Semantic feature tensors and contour feature tensors are obtained based on the enhanced road surface image and the contour image, respectively. A binarized segmentation map is obtained through the semantic feature tensor and the contour feature tensor, and the binarized segmentation map includes the diseased area. Based on the determined disease category and the binarized segmentation map, a road surface score is obtained. The road surface score is then compared with a score threshold to determine whether road maintenance is required.

2. The road maintenance determination method according to claim 1, characterized in that, The step of converting the initial road surface image into an enhanced illumination component and an enhanced reflection component includes: An initial optimization model is constructed, and a training input set is obtained, which includes training road surface images. The training input set is used as the input value of the initial optimization model, and the initial optimization model outputs training illumination output components and training reflection output components. Based on the training illumination output component and the training reflection output component, a reconstruction loss function, a reflection component constraint loss function, and an illumination component constraint loss function are constructed respectively, and a total loss function is constructed through the reconstruction loss function, the reflection component constraint loss function, and the illumination component constraint loss function. The initial optimization model is updated to the final optimization model using the total loss function, and the initial road surface image is used as the input value of the final optimization model to obtain the enhanced illumination component and the enhanced reflection component through the final optimization model.

3. The road maintenance determination method according to claim 2, characterized in that, The expression for the reconstruction loss function is: ， in, Represents the reconstruction loss function. Represents the training road surface image, This represents the training reflection output component. This represents the training illumination output component. This indicates element-wise multiplication. Represents the L1 norm; The expression for the reflection component constraint loss function is: ， in, Represents the reflection component constraint loss function. The V channel represents the training reflection output component. This represents the V channel of the training road surface image. Represents the HSV color space. Indicates channel extraction operation. This indicates an adaptive histogram equalization operation. Indicates the noise suppression coefficient. This represents the image gradient of the training reflection output component; The expression for the illumination component constraint loss function is: ， in, Represents the illumination component constraint loss function. This represents the image gradient of the training illumination output component. Represents the coupling coefficient. This represents an exponential function.

4. The road maintenance determination method according to claim 1, characterized in that, The step of obtaining the texture feature map corresponding to the enhanced road surface image and converting the texture feature map into a global feature sequence includes: The texture feature map of the enhanced road surface image is extracted using a CNN network, and the texture feature map includes several grid regions; In the spatial dimension, the texture feature map is flattened into a texture feature sequence, which includes several texture feature vectors corresponding to the grid region. Position encoding is added to the texture feature vectors to obtain several local feature vectors. The local feature vectors are updated to global feature vectors through a multi-head attention mechanism, and several of the global feature vectors constitute a global feature sequence.

5. The road maintenance determination method according to claim 4, characterized in that, The formula for obtaining the location code is: ， in, This represents the value of the position code in the 2i-th dimension. This represents the position code value in the (2i+1)th dimension. This represents the position index corresponding to the grid region, and 0 ≤ < ,in This indicates the total number of grid regions. Represents a dimension index, and 0 ≤ < , This represents the total dimension of the texture feature vector.

6. The road maintenance determination method according to claim 4, characterized in that, The step of obtaining the disease category corresponding to the global feature sequence includes: Dimensional equalization is performed on all the global feature vectors to obtain aggregate vectors; The aggregated vector is linearly transformed to convert it into a raw category score dataset, which includes several category scores that correspond to preset disease categories. Based on the category score, the disease probability corresponding to the preset disease category is obtained, and the preset disease category with the highest disease probability is selected as the disease category to be measured.

7. The road maintenance determination method according to claim 6, characterized in that, The formula for obtaining the category score is: ， in, This represents the category score corresponding to the i-th preset disease category. Represents an aggregate vector. Represents the weight matrix. This indicates the bias term.

8. The road maintenance determination method according to claim 1, characterized in that, The step of obtaining the binarized segmentation map using the semantic feature tensor and the contour feature tensor includes: The semantic feature tensor and the contour feature tensor are concatenated into a combined tensor, and a spatial attention map is obtained based on the combined tensor. The semantic feature tensor is weighted and modulated using the spatial attention map to obtain an enhanced feature tensor. The enhanced feature tensor is then upsampled to obtain a single-channel feature map with the same size as the enhanced road surface image. The original pixel value of each pixel in the single-channel feature map is mapped to a pixel probability. The pixel probability is compared with a probability threshold to determine the final pixel value of the pixel. The final pixel value is 0 or 1, so as to convert the single-channel feature map into a binary segmentation map.

9. The road maintenance determination method according to claim 1, characterized in that, The step of obtaining a pavement score based on the determined disease category and the binarized segmentation map includes: Obtain the disease weight corresponding to the measured disease category, and obtain the disease area of ​​the disease region; The pavement score is obtained based on the defect weight and the defect area.

10. A road maintenance discrimination system, applied to the road maintenance discrimination method as described in any one of claims 1 to 9, characterized in that, The system includes: The acquisition module is used to acquire an initial road surface image, convert the initial road surface image into an enhanced illumination component and an enhanced reflection component, and acquire an enhanced road surface image through the enhanced illumination component and the enhanced reflection component. The analysis module is used to obtain a texture feature map corresponding to the enhanced pavement image, convert the texture feature map into a global feature sequence, and obtain the measured disease category corresponding to the global feature sequence; An extraction module is used to perform edge detection on the enhanced road surface image to obtain a contour image, obtain a semantic feature tensor and a contour feature tensor based on the enhanced road surface image and the contour image respectively, and obtain a binarized segmentation map through the semantic feature tensor and the contour feature tensor, wherein the binarized segmentation map includes the diseased area. The execution module is used to obtain a pavement score based on the determined disease category and the binarized segmentation map, and compare the pavement score with a score threshold to determine whether road maintenance is required.

Images