A road maintenance assessment method and system based on machine vision

Abstract

The present invention relates to the field of image processing technology, and discloses a machine vision-based road maintenance assessment method and system. The method comprises: collecting image datasets at adjacent moments of a road to be inspected, determining three standard judgment images; obtaining the ambient light intensity at the time each standard judgment image was captured, and determining a restoration scheme for the standard judgment image; when the light intensity is below a light intensity threshold, determining the shadow area and performing image restoration using texture compensation reconstruction to obtain a restored judgment image; comparing one restored judgment image with the other two to determine constant and variable areas, extracting feature vectors of the variable areas, and performing cluster analysis with a preset road disease dataset; and performing road maintenance assessment and early warning based on the total area of ​​the diseased area. The present invention improves image quality and analysis accuracy in complex lighting environments through image standardization, ambient light intensity perception, and an adaptive image restoration strategy.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a road maintenance assessment method and system based on machine vision. Background Art

[0002] With the acceleration of urbanization, the pressure on urban roads continues to increase, making road maintenance and repair increasingly important. Traditional road inspection and maintenance rely on manual on-site inspections and assessments. Manual inspections are inefficient, labor-intensive, and easily affected by external factors such as weather and sunlight, resulting in inaccurate and delayed inspection data. Furthermore, manual inspections are highly subjective, and assessment results are influenced by the inspector's experience and judgment, lacking consistency and accuracy.

[0003] Invention patent CN115909256A is a road defect detection method based on road visual images, and proposes an automatic detection method based on image data. However, in this scheme, images are prone to feature misjudgment or missed detection under uneven lighting or complex lighting environments (such as shadows or strong light interference). Especially in low-light conditions, shadow areas are often mistakenly identified as defect areas, or the defect features are obscured, affecting the accuracy of judgment.

[0004] Therefore, it is necessary to design a road maintenance assessment method and system based on machine vision to solve the problems existing in current technology. Summary of the Invention

[0005] In view of this, the present invention proposes a road maintenance assessment method and system based on machine vision, aiming to solve the problems of low image detection accuracy and poor detection reliability in current road image detection.

[0006] In one aspect, the present invention provides a road maintenance assessment method based on machine vision, comprising:

[0007] An image acquisition device is used to collect image data sets of the road to be inspected at adjacent moments, wherein the image data sets include road images at at least three moments, and three of the road images are selected for image standardization to determine three standard judgment images.

[0008] respectively obtaining the ambient light intensity when each standard judgment image is captured, and determining a restoration scheme for the standard judgment image according to the ambient light intensity;

[0009] When the ambient light intensity is lower than the light intensity threshold, extracting the brightness component and the saturation component of each of the standard judgment images based on color space conversion, determining the shadow area based on the brightness change gradient and morphological analysis, and performing image restoration on the shadow area using texture compensation reconstruction to obtain a restored judgment image;

[0010] Comparing one of the repaired judgment images with the other two images, determining the constant area and the variable area of ​​each of the repaired judgment images based on the position comparison results, extracting the feature vector of the variable area, performing cluster analysis on the feature vector and a preset road disease dataset to determine whether the variable area is a disease area;

[0011] Road maintenance assessment and early warning are performed based on the total area of ​​the diseased areas in the constant area and the variable area.

[0012] Furthermore, selecting three road images and performing image standardization processing respectively to determine three standard judgment images includes:

[0013] Performing a standardization process on each of the road images, wherein the standardization process includes dividing the image size and normalizing the pixel values;

[0014] Inputting the normalized road image into a convolutional neural network model, wherein the convolutional neural network model includes multiple convolutional layers, pooling layers, and fully connected layers, and the fully connected layers are provided with a binary classification output layer;

[0015] Outputting an integrity value of the standardized road image based on the convolutional neural network model;

[0016] When the integrity value is greater than a preset integrity value, the road image is determined as the standard judgment image; when the integrity value is less than or equal to the preset integrity value, the road image is reselected and the integrity judgment is re-performed.

[0017] Furthermore, when determining the restoration scheme of the standard judgment image according to the ambient light intensity, it includes:

[0018] Comparing the ambient light intensity with an ambient light intensity threshold, and determining a restoration solution for the standard judgment image according to the comparison result;

[0019] When the ambient light intensity is greater than or equal to the ambient light intensity threshold, each of the standard judgment images is decomposed into three RGB channels, and the reflection component is obtained for each channel respectively. A multi-scale Retinex output is obtained based on weighted fusion, and finally the restored judgment image is obtained;

[0020] When the ambient light intensity is less than the ambient light intensity threshold, the brightness component and saturation component of each standard judgment image are extracted based on color space conversion, the shadow area is determined based on the brightness change gradient and morphological analysis, and the shadow area is repaired using texture compensation reconstruction to obtain a repaired judgment image.

[0021] Furthermore, each of the standard judgment images is decomposed into three channels of RGB, and the reflection component of each channel is obtained separately, including:

[0022]

[0023] in, represents the reflection component, Indicates the original pixel value of a channel in the standard judgment image, Represents a two-dimensional Gaussian function as the blur kernel for illumination component estimation, σ represents the blur degree control scale parameter, Represents the convolution operation.

[0024] Furthermore, after obtaining the multi-scale Retinex output based on the weighted fusion, the method further includes: performing color restoration on the multi-scale Retinex output result;

[0025]

[0026] in, represents the color restoration coefficient of the i-th channel, represents the pixel value of the i-th channel of the standard judgment image, α represents the gain control parameter, and β represents the dynamic range control parameter.

[0027] Furthermore, extracting the brightness component and saturation component of each standard judgment image based on color space conversion, and determining the shadow area based on brightness change gradient and morphological analysis includes:

[0028] Converting the standard judgment image into an HSV image and extracting the brightness component and saturation component;

[0029] Performing edge detection on the brightness component using a Sobel operator to determine an edge pixel area;

[0030] Determine each edge pixel as a shadow boundary when the brightness component of the edge pixel is less than a brightness component threshold and the saturation component is less than a saturation threshold;

[0031] An opening operation is performed on the shadow boundary to remove small noise points and a closing operation is performed to fill internal holes, and finally the shadow area is determined.

[0032] Furthermore, the shadow area is restored using texture compensation reconstruction to obtain a restored judgment image, which includes:

[0033] selecting a plurality of reference blocks in a non-shadow area adjacent to the shadow area;

[0034] The best reference block is selected based on the block similarity matching strategy;

[0035] mapping the content of the best reference block to the shadow area based on texture synthesis;

[0036] The brightness and color of the repaired area and the surrounding area are aligned based on local histogram matching to obtain the repaired judgment image.

[0037] Furthermore, extracting the characteristic vector of the change area and performing cluster analysis on the characteristic vector and a preset road disease data set includes:

[0038] The feature vector includes texture features, color and brightness features, and shape and geometry features;

[0039] Initialize the K value according to the disease type in the preset road disease data set, and iteratively search for K cluster centers, where the K value is the disease type plus 1;

[0040] Minimize the distance between the eigenvector of each of the change regions and the center of each cluster;

[0041] If the changed region is classified into the disease cluster with the closest distance, the changed region is determined to be the disease region; if the changed region is not classified into any disease cluster, the changed region is determined not to be the disease region.

[0042] Furthermore, when performing road maintenance assessment and early warning based on the total area of ​​the diseased areas in the constant area and the variable area, the method includes:

[0043] Comparing the total area of ​​the diseased area with a first preset area and a second preset area respectively, and performing a road maintenance assessment and early warning according to the comparison results; the first preset area is smaller than the second preset area;

[0044] When the total area of ​​the diseased area is less than or equal to the first preset area, a first-level road maintenance assessment warning is issued; when the total area of ​​the diseased area is greater than the first preset area and less than or equal to the second preset area, a second-level road maintenance assessment warning is issued; when the total area of ​​the diseased area is greater than the second preset area, a third-level road maintenance assessment warning is issued.

[0045] Compared with the prior art, the beneficial effects of the present invention are as follows: by collecting image data of the road to be inspected at multiple adjacent moments, and combining image standardization, ambient light intensity perception and adaptive image restoration strategies, the image quality and accuracy of subsequent analysis in complex lighting environments are improved. In order to solve the common problem of misjudgment of shadows under weak light conditions, color space conversion is used to extract brightness and saturation components, and the shadow area is accurately identified by combining brightness change gradient and morphological analysis. Texture compensation reconstruction restores occlusion information to ensure the complete preservation of image defect features. Constant areas and variable areas are identified by multi-moment image comparison, and multi-dimensional feature vectors are extracted for the variable areas. Cluster analysis is performed in combination with a preset disease data set to achieve accurate determination of the diseased area, avoiding errors from manual experience and subjective influences. Finally, the road maintenance grade assessment and risk warning are completed based on the area distribution of the diseased area, overcoming the problems of large recognition errors and inconsistent judgments under light interference in the prior art.

[0046] On the other hand, the present application also provides a machine vision-based road maintenance assessment system for applying the above-mentioned machine vision-based road maintenance assessment method, comprising:

[0047] an acquisition unit configured to acquire an image data set of the road to be inspected at adjacent moments through an image acquisition device, wherein the image data set includes road images at at least three moments, select three of the road images, perform image standardization processing on each of the three images, and determine three standard judgment images;

[0048] A judgment unit is configured to respectively obtain the ambient light intensity when each standard judgment image is captured, and determine a restoration solution for the standard judgment image according to the ambient light intensity;

[0049] a processing unit configured to, when the ambient light intensity is lower than a light intensity threshold, extract a brightness component and a saturation component of each of the standard judgment images based on color space conversion, determine a shadow area based on a brightness change gradient and morphological analysis, and perform image restoration on the shadow area using texture compensation reconstruction to obtain a restored judgment image;

[0050] a verification unit configured to compare one of the restored judgment images with the other two images, determine a constant area and a variable area of ​​each of the restored judgment images based on the position comparison results, extract a feature vector of the variable area, perform cluster analysis on the feature vector and a preset road disease dataset, and determine whether the variable area is a disease area;

[0051] The early warning unit is configured to perform road maintenance assessment and early warning based on the total area of ​​the diseased areas in the constant area and the variable area.

[0052] It is understandable that the above-mentioned machine vision-based road maintenance assessment method and system have the same beneficial effects and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present invention. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0054] Figure 1 A flowchart of a machine vision-based road maintenance assessment method provided in an embodiment of the present invention;

[0055] Figure 2 This is a structural block diagram of a machine vision-based road maintenance assessment system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0056] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, unless there is a conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0057] In some embodiments of the present application, see Figure 1 As shown, a road maintenance assessment method based on machine vision includes:

[0058] S100: An image acquisition device is used to collect image data sets of a road to be detected at adjacent moments, wherein the image data sets include road images at at least three moments, and three road images are selected for image standardization to determine three standard judgment images.

[0059] S200: The ambient light intensity at the time each standard judgment image was captured is obtained, and a restoration scheme for the standard judgment image is determined based on the ambient light intensity. When the ambient light intensity is below a light intensity threshold, the brightness and saturation components of each standard judgment image are extracted using color space conversion. Shadow areas are determined based on brightness gradient and morphological analysis, and texture compensation reconstruction is used to restore the shadow areas to obtain the restored judgment image.

[0060] S300: Compare one of the repaired judgment images with the other two images, determine the constant area and the variable area of ​​each repaired judgment image based on the position comparison results, extract the feature vector of the variable area, perform cluster analysis on the feature vector and the preset road disease data set, and determine whether the variable area is a disease area.

[0061] S400: Performing road maintenance assessment and early warning based on the total area of ​​the diseased areas in the constant area and the variable area.

[0062] Specifically, in step S100, an image acquisition device (such as an industrial camera, vehicle-mounted camera system, or drone) collects image data of the road to be inspected at at least three adjacent time points to form an image sequence. These images undergo image standardization, such as resizing, brightness normalization, and geometric alignment, to ensure consistency and accuracy in subsequent image comparison. Three representative "standard judgment images" are then selected as the core analysis targets. In step S200, the ambient light intensity at the time each standard judgment image was captured is obtained using a light sensor or global image brightness estimation method. This intensity is then compared with a preset light intensity threshold, and an appropriate image restoration strategy is selected based on the comparison result. If the image is in a low-light environment, the RGB image is first converted to HSV or Lab color space, and the brightness component (such as V or L) and saturation component are extracted. Then, shadow regions in the image are identified and segmented by combining brightness gradient analysis with morphological analysis methods (such as opening and closing operations). For the identified shadow regions, texture compensation and reconstruction algorithms, such as non-local texture interpolation based on neighborhood blocks, are used to restore the occluded image content, resulting in a restored judgment image. This effectively mitigates misidentification of defects due to shadows or insufficient lighting. In step S300, a reference image is selected from the restored images and compared with the other two images in time series. Using feature alignment and motion estimation techniques (such as SURF / SIFT keypoint registration), pixel position changes are used to identify constant regions (long-term stable structures) and variable regions (recent structural changes). Feature vectors are extracted for the variable regions, typically including grayscale texture features, color statistics, and edge contour features. Cluster analysis, such as using K-means, is performed on the extracted feature vectors and a pre-set dataset of road damage (e.g., annotated image data of cracks, potholes, and fissures) to determine whether the variable regions represent actual road damage. In step S400, a road maintenance assessment grade or early warning is assigned based on the total area of ​​the identified defect regions within the constant and variable regions, assisting the operations department in precise repairs and resource scheduling.

[0063] Understandably, to address the low efficiency, significant lighting interference, and easy confusion between shadows and defects associated with traditional manual inspections, an assessment process with adaptive image restoration and intelligent identification capabilities has been developed. Through ambient light intensity perception, color space analysis, and image texture reconstruction, image quality stability is ensured under complex lighting conditions. Combined with multi-time comparison and cluster analysis, high-precision identification and intelligent classification of defect areas are achieved. Area quantification and grade assessment provide a basis for decision-making in road maintenance, improving the automation, accuracy, and real-time nature of road defect identification.

[0064] In some embodiments of the present application, three road images are selected and subjected to image standardization processing respectively. When determining three standard judgment images, the process includes: performing standardization processing on each road image, wherein the standardization processing includes segmenting the image size and normalizing the pixel values. The standardized road image is input into a convolutional neural network model, wherein the convolutional neural network model includes multiple convolutional layers, pooling layers, and fully connected layers, and a binary classification output layer is provided in the fully connected layer. The integrity value of the standardized road image is output based on the convolutional neural network model. When the integrity value is greater than a preset integrity value, the road image is determined as a standard judgment image. When the integrity value is less than or equal to the preset integrity value, a new road image is selected and the integrity judgment is performed again.

[0065] Specifically, each raw road image undergoes standardization preprocessing, which involves two steps: image resizing: unifying the image to a specific size (such as 512×512 or 224×224) to eliminate spatial resolution differences caused by different acquisition devices or shooting angles; and pixel value normalization: converting the original pixel values ​​(typically 01) to conform to the input data distribution requirements of deep learning models, ensuring consistency between training and inference. The normalized images are then input into a convolutional neural network model, which consists of multiple convolutional layers (to extract local spatial features), pooling layers (to reduce dimensionality and enhance invariance), and fully connected layers (to combine features and make classification decisions). The final fully connected layer is equipped with a binary classification output layer, which determines whether the input image is complete and outputs a completeness value (such as a confidence probability). The integrity value output by the model is compared with the preset threshold. If the value is greater than the preset integrity value threshold (such as 0.85), it means that the image is complete, clear, without strong interference or large areas of missing images, and can be used as a "standard judgment image" for subsequent analysis; if it is less than or equal to the threshold, it is determined that the image has occlusion, strong light interference, blur or defects, and the image needs to be discarded and the next image must be selected from the image dataset for integrity judgment until three qualified images are selected.

[0066] In some embodiments of the present application, when determining a standard for determining an image restoration solution based on ambient light intensity, the method includes comparing the ambient light intensity with an ambient light intensity threshold, and determining a standard for determining an image restoration solution based on the comparison result.

[0067] Specifically, when the ambient light intensity is greater than or equal to the ambient light intensity threshold, each standard judgment image is decomposed into three RGB channels, and the reflection component of each channel is obtained separately. Based on weighted fusion, a multi-scale Retinex output is obtained, and finally the restored judgment image is obtained. When the ambient light intensity is less than the ambient light intensity threshold, the brightness and saturation components of each standard judgment image are extracted based on color space conversion. The shadow area is determined based on the brightness change gradient and morphological analysis. The shadow area is then restored using texture compensation reconstruction to obtain the restored judgment image.

[0068] In some embodiments of the present application, each standard judgment image is decomposed into three RGB channels, and the reflection component of each channel is obtained separately, including:

[0069]

[0070] in, represents the reflection component, Indicates the original pixel value of a channel in the standard judgment image, Represents a two-dimensional Gaussian function as the blur kernel for illumination component estimation, σ represents the blur degree control scale parameter, Represents the convolution operation.

[0071] In some embodiments of the present application, after obtaining the multi-scale Retinex output based on weighted fusion, the method further includes: performing color restoration on the multi-scale Retinex output result.

[0072]

[0073] in, represents the color restoration coefficient of the i-th channel, represents the pixel value of the i-th channel of the standard judgment image, α represents the gain control parameter, and β represents the dynamic range control parameter. The value range of α is 100-150. The value range of β is 20-50.

[0074] Specifically, .

[0075] As you can understand, (x, y) in the above calculations represents the spatial coordinate position in the image, namely the horizontal (column) and vertical (row) coordinates of the currently calculated pixel in the image. After decomposing each standard judgment image into three color channels (R, G, and B), the reflectance component is calculated for each channel. Because single-scale Retinex can easily lead to over-enhancement or distortion of details, a multi-scale fusion mechanism is introduced. By setting different σ values ​​(e.g., 15, 80, 250), multiple reflectance component images are generated, which are then fused in a weighted manner to form the final output. Multi-scale Retinex can easily disrupt the color balance of the original image, so a color restoration function is introduced to correct this. By splitting the standard judgment image into RGB channels, Retinex theory is used to separate the reflectance component from each channel. A multi-scale fusion strategy is then used to enhance image details. A color restoration mechanism is then introduced to effectively prevent color casts and false colors. This improves the visual clarity and feature integrity of images under complex lighting conditions, ensuring the natural color and structural consistency of the enhanced image, providing a stable and reliable image foundation for accurate identification and intelligent judgment of diseased areas.

[0076] In some embodiments of the present application, the brightness and saturation components of each standard judgment image are extracted based on color space conversion, and the shadow area is determined based on brightness change gradient and morphological analysis, including: converting the standard judgment image into an HSV image and extracting the brightness and saturation components. Using the Sobel operator to perform edge detection on the brightness component to determine the edge pixel area. Each edge pixel is judged, and when the brightness component of the edge pixel is less than the brightness component threshold and the saturation component is less than the saturation threshold, the edge pixel is determined to be a shadow boundary. An opening operation is performed on the shadow boundary to remove small noise points and a closing operation is performed to fill the internal holes, and finally the shadow area is determined.

[0077] In some embodiments of the present application, using texture compensation reconstruction to perform image restoration on a shadow area and obtain a restored judgment image includes: selecting several reference blocks in a non-shadow area adjacent to the shadow area; selecting the best reference block based on a block similarity matching strategy; mapping the content of the best reference block to the shadow area based on texture synthesis; and aligning the brightness and color of the restored area with the surrounding area based on local histogram matching to obtain a restored judgment image.

[0078] Specifically, the standard judgment image is converted from RGB color space to HSV (Hue, Saturation, Value) space. The brightness (V) and saturation (S) components are extracted for subsequent analysis. HSV space effectively distinguishes color and brightness information, facilitating the location of areas with abnormal lighting without affecting color structure. The Sobel edge detection operator is used to calculate the gradient of the brightness component to obtain edge pixels in the image, thereby locating potential shadow boundaries. For each edge pixel, its brightness and saturation components are compared: if the brightness value is less than a set brightness threshold and the saturation value is less than the saturation threshold, the pixel is considered to be at the shadow edge. An opening operation is used to remove isolated small noise points (i.e., erosion followed by dilation), followed by a closing operation (i.e., dilation followed by erosion) to fill in the edge contours, improving the connectivity and integrity of the shadow area and accurately outputting the shadow mask. Beyond the identified shadow area, the adjacent non-shadow area is selected as a reference area and divided into several texture blocks for subsequent texture compensation. Using block similarity matching strategies, such as those based on the L2 norm or SSIM structural similarity, a reference block is selected that most closely resembles the local structure of the shadow area to ensure the consistency and authenticity of the compensated texture. Texture synthesis techniques (such as pixel fusion or the PatchMatch algorithm) are then used to map the content of the optimal reference block to the shadow area to achieve texture restoration. To avoid sudden brightness changes or color inconsistencies, local histogram matching is further performed between the inpainted area and its surrounding areas to achieve a smooth alignment of brightness and color styles.

[0079] It is understandable that by combining the HSV color space with an edge detection mechanism, shadow areas in the image caused by low light or uneven illumination are extracted, and multi-level morphological operations are used to improve the accuracy and consistency of shadow positioning. During the shadow restoration stage, a texture compensation method based on texture block similarity matching and a brightness and color alignment mechanism are introduced. This ensures that the restored image details are realistic and the texture is coherent, allowing it to naturally integrate into the overall picture, improving the image usability in complex lighting scenarios.

[0080] In some embodiments of the present application, feature vectors of the changed regions are extracted and clustered with a preset road defect dataset. The feature vectors include texture features, color and brightness features, and shape and geometry features. A K value is initialized based on the defect type in the preset road defect dataset, and K cluster centers are iteratively searched, where the K value is the defect type plus 1. The distance between the feature vector of each changed region and the center of each cluster is minimized. If the changed region is classified into the nearest defect cluster, the changed region is determined to be a defect region. If the changed region is not classified into any defect cluster, the changed region is determined not to be a defect region.

[0081] In some embodiments of the present application, when performing a road maintenance assessment and early warning based on the total area of ​​the damaged area within the constant area and the variable area, the method includes comparing the total area of ​​the damaged area with a first preset area and a second preset area, respectively, and performing a road maintenance assessment and early warning based on the comparison results. The first preset area is smaller than the second preset area.

[0082] Specifically, when the total area of ​​the damaged area is less than or equal to the first preset area, a Level 1 road maintenance assessment warning is issued. When the total area of ​​the damaged area is greater than the first preset area and less than or equal to the second preset area, a Level 2 road maintenance assessment warning is issued. When the total area of ​​the damaged area is greater than the second preset area, a Level 3 road maintenance assessment warning is issued.

[0083] It is understandable that an unsupervised clustering algorithm similar to K-means is used to classify the changed areas. The K value is set according to the number of disease types contained in the road disease dataset, and K is set to "the number of disease types + 1", where the additional "+1" is used to represent the non-disease class. The center of each cluster is iteratively calculated in the high-dimensional feature space, and the classification is completed by minimizing the distance between the region vector and the center of each cluster; if the feature vector of a changed area is closest to the center of a disease cluster, it is identified as a disease area, otherwise it is a non-disease area. By constructing a multi-dimensional feature vector and combining it with historical disease data for cluster identification, the identification accuracy and type adaptability of the disease area are improved, and the problem of difficulty in distinguishing between disease and non-disease areas in complex backgrounds is solved. The multi-level warning mechanism based on the area of ​​the diseased area establishes a quantitative standard for road risk assessment, which can dynamically adjust the maintenance priority according to the degree of disease, realizing the transition from "passive response" to "active warning".

[0084] In the above embodiment, by collecting image data of the road to be inspected at multiple adjacent moments, and combining image standardization, ambient light intensity perception and adaptive image restoration strategies, the image quality and accuracy of subsequent analysis in complex lighting environments are improved. In order to address the common problem of misjudgment of shadows under weak light conditions, color space conversion is used to extract brightness and saturation components, and the shadow area is accurately identified by combining brightness change gradient and morphological analysis. Texture compensation is then used to reconstruct and restore occlusion information to ensure the complete preservation of image defect features. Constant areas and variable areas are identified through multi-moment image comparison, and multi-dimensional feature vectors are extracted for the variable areas. Cluster analysis is performed in combination with a preset defect data set to achieve accurate determination of the defect area, avoiding errors from manual experience and subjective influences. Finally, the road maintenance level assessment and risk warning are completed based on the area distribution of the defect area, overcoming the problems of large recognition errors and inconsistent judgments in the existing technology under light interference.

[0085] In another preferred embodiment based on the above embodiment, refer to Figure 2As shown, this embodiment provides a machine vision-based road maintenance assessment system for applying the above-mentioned machine vision-based road maintenance assessment method, including:

[0086] The acquisition unit is configured to acquire an image data set of the road to be detected at adjacent moments through an image acquisition device, wherein the image data set includes road images at at least three moments, selects three road images for image standardization processing respectively, and determines three standard judgment images.

[0087] The judgment unit is configured to respectively obtain the ambient light intensity when each standard judgment image is captured, and determine a restoration solution for the standard judgment image according to the ambient light intensity.

[0088] The processing unit is configured to extract the brightness component and saturation component of each standard judgment image based on color space conversion when the ambient light intensity is lower than the light intensity threshold, determine the shadow area based on the brightness change gradient and morphological analysis, and use texture compensation reconstruction to perform image restoration on the shadow area to obtain a restored judgment image.

[0089] The verification unit is configured to compare one of the repaired judgment images with the other two images, determine the constant area and the variable area of ​​each repaired judgment image based on the position comparison result, extract the feature vector of the variable area, perform cluster analysis on the feature vector and a preset road disease data set, and determine whether the variable area is a disease area.

[0090] The early warning unit is configured to perform road maintenance assessment and early warning based on the total area of ​​the diseased area in the constant area and the variable area.

[0091] It is understandable that by collecting image data of the road to be inspected at multiple adjacent moments, and combining image standardization, ambient light intensity perception and adaptive image restoration strategies, the image quality and accuracy of subsequent analysis in complex lighting environments are improved. In order to address the common problem of misjudgment of shadows under weak light conditions, color space conversion is used to extract brightness and saturation components, and the shadow area is accurately identified by combining brightness change gradient and morphological analysis. Texture compensation is then used to reconstruct and restore occlusion information to ensure the complete preservation of image defect features. Constant areas and variable areas are identified through multi-moment image comparison, and multi-dimensional feature vectors are extracted for the variable areas. Cluster analysis is performed in combination with a preset defect data set to achieve accurate determination of the defect area, avoiding errors from manual experience and subjective influences. Finally, the road maintenance level assessment and risk warning are completed based on the area distribution of the defect area, overcoming the problems of large recognition errors and inconsistent judgments in existing technologies under light interference.

[0092] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0093] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0094] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0095] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. A road maintenance assessment method based on machine vision, characterized in that: include: An image acquisition device is used to collect image data sets of the road to be inspected at adjacent moments, wherein the image data sets include road images at at least three moments, and three of the road images are selected for image standardization to determine three standard judgment images. respectively obtaining the ambient light intensity when each standard judgment image is captured, and determining a restoration scheme for the standard judgment image according to the ambient light intensity; When the ambient light intensity is lower than the light intensity threshold, extracting the brightness component and the saturation component of each of the standard judgment images based on color space conversion, determining the shadow area based on the brightness change gradient and morphological analysis, and performing image restoration on the shadow area using texture compensation reconstruction to obtain a restored judgment image; Comparing one of the restored judgment images with the other two images, determining the constant area and the variable area of ​​each of the restored judgment images based on the position comparison results, extracting the feature vector of the variable area, performing cluster analysis on the feature vector and a preset road disease dataset to determine whether the variable area is a disease area; Performing road maintenance assessment and early warning based on the total area of ​​the diseased areas within the constant area and the variable area; Selecting the three road images and performing image standardization processing respectively to determine the three standard judgment images includes: Performing a standardization process on each of the road images, wherein the standardization process includes segmenting the image size and normalizing the pixel values; Inputting the normalized road image into a convolutional neural network model, wherein the convolutional neural network model includes multiple convolutional layers, pooling layers, and fully connected layers, and the fully connected layers are provided with a binary classification output layer; Outputting an integrity value of the standardized road image based on the convolutional neural network model; When the integrity value is greater than a preset integrity value, the road image is determined as the standard judgment image; when the integrity value is less than or equal to the preset integrity value, the road image is reselected and the integrity judgment is re-performed; When determining the restoration scheme of the standard judgment image according to the ambient light intensity, the method includes: Comparing the ambient light intensity with an ambient light intensity threshold, and determining a restoration solution for the standard judgment image according to the comparison result; When the ambient light intensity is greater than or equal to the ambient light intensity threshold, each of the standard judgment images is decomposed into three RGB channels, and the reflection component is obtained for each channel respectively. A multi-scale Retinex output is obtained based on weighted fusion, and finally the restored judgment image is obtained; When the ambient light intensity is less than the ambient light intensity threshold, the brightness component and saturation component of each standard judgment image are extracted based on color space conversion, the shadow area is determined based on the brightness change gradient and morphological analysis, and the shadow area is repaired using texture compensation reconstruction to obtain a repaired judgment image.

2. The machine vision-based road maintenance assessment method according to claim 1, characterized in that: Decomposing each of the standard judgment images into three RGB channels and obtaining the reflection component of each channel respectively includes: in, represents the reflection component, Indicates the original pixel value of a channel in the standard judgment image, Represents a two-dimensional Gaussian function as the blur kernel for illumination component estimation, σ represents the blur degree control scale parameter, and ∗ represents the convolution operation.

3. The machine vision-based road maintenance assessment method according to claim 2, characterized in that: After obtaining the multi-scale Retinex output based on weighted fusion, the method further includes: performing color restoration on the multi-scale Retinex output result; in, represents the color restoration coefficient of the i-th channel, represents the pixel value of the i-th channel of the standard judgment image, α represents the gain control parameter, and β represents the dynamic range control parameter.

4. The machine vision-based road maintenance assessment method according to claim 1, characterized in that: Extracting the brightness component and saturation component of each standard judgment image based on color space conversion, and determining the shadow area based on brightness change gradient and morphological analysis, including: Converting the standard judgment image into an HSV image and extracting the brightness component and saturation component; Performing edge detection on the brightness component using a Sobel operator to determine an edge pixel area; Determine each edge pixel as a shadow boundary when the brightness component of the edge pixel is less than a brightness component threshold and the saturation component is less than a saturation threshold; An opening operation is performed on the shadow boundary to remove small noise points and a closing operation is performed to fill internal holes, and finally the shadow area is determined.

5. The machine vision-based road maintenance assessment method according to claim 4, characterized in that: Performing image restoration on the shadow area using texture compensation reconstruction to obtain a restored judgment image includes: selecting a plurality of reference blocks in a non-shadow area adjacent to the shadow area; The best reference block is selected based on the block similarity matching strategy; mapping the content of the best reference block to the shadow area based on texture synthesis; The brightness and color of the repaired area and the surrounding area are aligned based on local histogram matching to obtain the repaired judgment image.

6. The machine vision-based road maintenance assessment method according to claim 5, characterized in that: Extracting the characteristic vector of the change area and performing cluster analysis on the characteristic vector and a preset road disease data set includes: The feature vector includes texture features, color and brightness features, and shape and geometry features; Initialize the K value according to the disease type in the preset road disease data set, and iteratively search for K cluster centers, where the K value is the disease type plus 1; Minimize the distance between the eigenvector of each of the change regions and the center of each cluster; If the changed region is classified into the disease cluster with the closest distance, the changed region is determined to be the disease region; if the changed region is not classified into any disease cluster, the changed region is determined not to be the disease region.

7. The machine vision-based road maintenance assessment method according to claim 6, characterized in that: When conducting road maintenance assessment and early warning based on the total area of ​​the diseased areas in the constant area and the variable area, it includes: Comparing the total area of ​​the diseased area with a first preset area and a second preset area respectively, and performing a road maintenance assessment and early warning according to the comparison results; the first preset area is smaller than the second preset area; When the total area of ​​the diseased area is less than or equal to the first preset area, a first-level road maintenance assessment warning is issued; when the total area of ​​the diseased area is greater than the first preset area and less than or equal to the second preset area, a second-level road maintenance assessment warning is issued; when the total area of ​​the diseased area is greater than the second preset area, a third-level road maintenance assessment warning is issued.

8. A machine vision-based road maintenance assessment system, for applying the machine vision-based road maintenance assessment method according to any one of claims 1 to 7, characterized in that: include: an acquisition unit configured to acquire an image data set of the road to be inspected at adjacent moments through an image acquisition device, wherein the image data set includes road images at at least three moments, select three of the road images, perform image standardization processing on each of the three images, and determine three standard judgment images; A judgment unit is configured to respectively obtain the ambient light intensity when each standard judgment image is captured, and determine a restoration solution for the standard judgment image according to the ambient light intensity; a processing unit configured to, when the ambient light intensity is lower than a light intensity threshold, extract a brightness component and a saturation component of each of the standard judgment images based on color space conversion, determine a shadow area based on a brightness change gradient and morphological analysis, and perform image restoration on the shadow area using texture compensation reconstruction to obtain a restored judgment image; a verification unit configured to compare one of the restored judgment images with the other two images, determine a constant area and a variable area of ​​each of the restored judgment images based on the position comparison results, extract a feature vector of the variable area, perform cluster analysis on the feature vector and a preset road disease dataset, and determine whether the variable area is a disease area; The early warning unit is configured to perform road maintenance assessment and early warning based on the total area of ​​the diseased areas in the constant area and the variable area.

Images