A method for identifying road surface collapse based on self-luminous paint

Abstract

This invention discloses a method for identifying road surface collapse based on self-luminous coating, comprising: acquiring an image of a road marking area using a camera device; extracting the marking area after preprocessing the image; analyzing the integrity of the marking from the dimensions of continuity, brightness, and width to obtain continuity anomaly, brightness anomaly, and width anomaly; verifying the temporal consistency of abnormal areas in multiple consecutive frames of images to obtain temporal confidence; calculating a comprehensive collapse confidence based on a weighted average of each anomaly and temporal confidence; and determining that a road surface collapse exists in the corresponding area and outputting the detection result when the comprehensive collapse confidence is greater than or equal to a preset threshold.

Description

Technical Field

[0001] This invention relates to the field of road surface detection technology, and in particular to a method for identifying road surface collapse based on self-luminous coating. Background Technology

[0002] Road subsidence detection is a crucial technical means to ensure the safe operation of urban roads. Currently, relevant detection methods mainly include manual inspection, physical detection, and image-based recognition technology. Early methods relied heavily on manual inspection, judging by observing road surface cracks and settlement, but this approach was inefficient and heavily influenced by subjective factors. With technological advancements, equipment such as ground-penetrating radar and displacement sensors were gradually introduced to detect the internal structure and changes of the roadbed. These methods can obtain relatively accurate underground information, but the equipment is costly and the implementation process is relatively complex. In recent years, detection methods based on image processing and machine learning have been applied, analyzing road surface images to identify subsidence areas, thus improving detection efficiency and automation to some extent.

[0003] However, the aforementioned methods still have significant shortcomings in practical applications. Video image-based detection methods are highly dependent on ambient lighting conditions; image quality deteriorates significantly at night or in low-light environments, resulting in insufficient contrast between the markings and the background, which can easily lead to reduced recognition accuracy or even missed detections. Although there are existing technical solutions for applying strontium aluminate self-luminescent materials to road markings, current self-luminescent markings are only used for nighttime illumination or driving guidance and have not yet been used for pavement structural health monitoring. In other words, current technology lacks a technical approach that can utilize the self-luminescent properties of road markings to characterize the pavement structural condition, thereby achieving low-cost passive detection.

[0004] In summary, there is an urgent need for a road collapse detection method that can reliably identify road collapses under nighttime lighting conditions without relying on expensive sensing equipment. Summary of the Invention

[0005] The problem addressed in this application is how to develop a road collapse detection method that can reliably identify road collapses under different lighting conditions without relying on expensive sensing equipment.

[0006] This application provides a method for identifying road subsidence based on self-luminous coating, applied to a road subsidence detection system. The road subsidence detection system includes a camera device and a processing module disposed on one side of the road surface. The surface of the road surface contains a marking coating of self-luminous material, which absorbs light energy during the day and continuously emits light at night. The method includes:

[0007] The camera device acquires images of the road marking area;

[0008] The image is preprocessed to obtain a preprocessed image;

[0009] Based on the preprocessed image, extract the marking region;

[0010] Based on the extracted marking regions, the integrity of the markings is analyzed from the dimensions of continuity, brightness, and width, respectively, to obtain the continuity anomaly, brightness anomaly, and width anomaly.

[0011] Temporal consistency verification is performed on abnormal regions detected in multiple consecutive frames of the image to obtain temporal confidence.

[0012] The comprehensive collapse confidence is obtained by weighting the continuity anomaly, the brightness anomaly, the width anomaly, and the temporal confidence.

[0013] When the overall collapse confidence level is greater than or equal to the preset collapse judgment threshold, it is determined that there is a road collapse in the corresponding area, and the detection result is output.

[0014] Furthermore, the preprocessing of the image to obtain a preprocessed image includes:

[0015] The image is converted to grayscale to obtain a grayscale image;

[0016] Based on a preset pixel range for the road surface area, the region of interest (ROI) is extracted from the grayscale image to obtain an ROI image.

[0017] The ROI region image is filtered and denoised to obtain a denoised image;

[0018] The denoised image is subjected to adaptive histogram equalization to obtain the preprocessed image.

[0019] Furthermore, the step of filtering and denoising the ROI region image to obtain a denoised image includes:

[0020] The ROI region image is denoised using a Gaussian filtering method.

[0021] The Gaussian filtering method is based on a Gaussian distribution function to construct a filter kernel, and performs a weighted average operation on each pixel and its neighboring pixels to obtain the denoised image.

[0022] Furthermore, the step of filtering and denoising the ROI region image to obtain a denoised image includes:

[0023] The ROI region image is denoised using a bilateral filtering method.

[0024] The bilateral filtering method constructs weights based on the spatial distance and gray-level similarity between pixels, and performs a weighted average operation on each pixel and its neighboring pixels to obtain the denoised image.

[0025] Furthermore, the step of extracting the marking region based on the preprocessed image includes:

[0026] The preprocessed image is then subjected to Sobel edge enhancement processing to obtain an edge-enhanced image;

[0027] Based on the edge-enhanced image, line segments are extracted using the LSD line segment detection algorithm to obtain an initial set of line segments;

[0028] The initial set of line segments is filtered by line segment length threshold and line segment tilt angle range to obtain a preliminary set of effective line segments;

[0029] The Cohen-Sutherland line segment clipping algorithm is used to clip the ROI region of the preliminary set of valid line segments to obtain the final set of valid line segments.

[0030] The marking area is determined based on the final set of valid marking segments.

[0031] Furthermore, based on the edge-enhanced image, line segments are extracted using the LSD line segment detection algorithm to obtain an initial line segment set, including:

[0032] Based on the grayscale gradients of each pixel in the horizontal and vertical directions obtained during the Sobel edge enhancement process, the gradient magnitude and gradient direction angle of each pixel are calculated.

[0033] Set a gradient threshold, retain pixels with gradient magnitude greater than or equal to the gradient threshold, and filter background noise pixels with gradient magnitude lower than the gradient threshold;

[0034] Wherein, the gradient threshold is a preset percentage of the maximum gradient magnitude in the edge enhancement image;

[0035] Based on the retained pixels, a direction angle similarity threshold is set, and adjacent pixels among the retained pixels whose gradient direction angle difference is less than or equal to the direction angle similarity threshold are aggregated into line segment support regions.

[0036] The minimum bounding rectangle is fitted to the support region of each line segment to obtain the starting point coordinates, ending point coordinates, inclination angle and length of each line segment;

[0037] The initial set of line segments is generated based on the starting point coordinates, the ending point coordinates, the inclination angle, and the length of each line segment.

[0038] Furthermore, the step of extracting the marking region based on the preprocessed image includes:

[0039] The preprocessed image is normalized to obtain a normalized image;

[0040] The normalized image is input into a pre-trained semantic segmentation network to obtain a binarized segmentation mask image, wherein the region with a pixel value of 1 in the binarized segmentation mask image represents the marking region, and the region with a pixel value of 0 represents the road background region.

[0041] Perform morphological closing operation on the binarized segmented mask image to obtain a regularized mask image;

[0042] The morphological closing operation is a composite operation that sequentially performs dilation and erosion operations;

[0043] The marking region is determined based on the regularized mask image.

[0044] Furthermore, 8. The analysis of the integrity of the markings from the continuity dimension to obtain the continuity anomaly degree includes:

[0045] When the system is first deployed, normal images of undamaged road markings are collected;

[0046] Based on the normal image, the single-pixel centerline skeleton of the marking is extracted using a morphological thinning algorithm;

[0047] The skeleton along the centerline is divided into several continuous small segments by a preset fixed step size.

[0048] The arc length of each of the continuous small segments is calculated segment by segment to obtain the standard length sequence of the arc segment, and the standard length sequence of the arc segment is stored in a fixed manner.

[0049] During subsequent detection, the same morphological thinning algorithm is used to extract the centerline skeleton of the current frame line, and it is segmented along the same arc path with the fixed step size, and the actual arc length of each segment is calculated segment by segment;

[0050] The actual arc length of each segment is compared with the standard arc length of the corresponding segment in the standard arc length sequence;

[0051] When the actual arc length is less than the corresponding standard arc length, the length loss rate of that segment is obtained by the ratio of the difference between the standard arc length and the actual arc length to the standard arc length.

[0052] When the actual arc length is greater than or equal to the corresponding standard arc length, the length loss rate of that segment is set to zero.

[0053] Segments with a length loss rate greater than or equal to a preset continuity anomaly threshold are identified as local continuity anomaly segments.

[0054] The proportion of the locally discontinuous segments to the total number of segments is calculated, and this proportion is used as the degree of discontinuity.

[0055] Furthermore, the analysis of the integrity of the markings from the brightness dimension to obtain the brightness anomaly degree includes:

[0056] Calculate the overall average brightness of the marked area;

[0057] The sliding window of a preset size is used to traverse the marked area row by row and column by column, and the area covered by each sliding window is taken as a local sub-region.

[0058] For each of the aforementioned local sub-regions, calculate the average brightness of each local sub-region;

[0059] Based on the overall average brightness and the total number of pixels in the local sub-region, calculate the average brightness of the caliper after removing the current local sub-region, and use it as the reference brightness value of the local sub-region.

[0060] The local brightness anomaly degree of each local sub-region is obtained based on the ratio of the average brightness of the local sub-region to the corresponding reference brightness value;

[0061] Local sub-regions with a local brightness anomaly level greater than or equal to a preset brightness anomaly threshold are identified as brightness anomaly regions.

[0062] The proportion of the brightness anomaly region to the total number of local sub-regions is calculated, and this proportion is taken as the brightness anomaly degree.

[0063] The analysis of the integrity of the grading line from the width dimension to obtain the width anomaly degree includes:

[0064] When the system is first deployed, normal images of undamaged road markings are collected;

[0065] Based on the normal image, the pixel width of the markings in the region of interest is counted line by line, and the average width of each line of the normal image in multiple consecutive frames is taken to obtain a row-level width standard value sequence, and the row-level width standard value sequence is stored in a fixed manner.

[0066] During subsequent detection, the actual width of the marking line is calculated line by line for the current frame image;

[0067] The actual width of each row is compared with the standard width value of the corresponding row in the row-level width standard value sequence;

[0068] When the actual width is less than the corresponding standard width value, the width deviation rate of the row is obtained by the ratio of the difference between the standard width value and the actual width to the standard width value.

[0069] When the actual width is greater than or equal to the corresponding standard width value, the width deviation rate of the row is set to zero;

[0070] Rows with a width deviation rate greater than or equal to a preset width anomaly threshold are identified as width anomaly rows;

[0071] The proportion of rows with abnormal widths to the total number of rows is calculated, and this proportion is used as the width abnormality degree.

[0072] The step of performing temporal consistency verification on abnormal regions detected in multiple consecutive frames of images to obtain temporal confidence includes:

[0073] Obtain the abnormal regions detected in each frame of the image across multiple consecutive frames;

[0074] Calculate the intersection pixel area of ​​the abnormal regions in each of the consecutive frames;

[0075] The spatiotemporal overlap rate is obtained by the ratio of the area of ​​the intersection pixels to the area of ​​the pixel region of the abnormal region in the initial frame.

[0076] The spatiotemporal overlap rate is compared with a preset overlap rate threshold.

[0077] When the spatiotemporal overlap rate is less than the overlap rate threshold, the temporal confidence is set to zero.

[0078] When the spatiotemporal overlap rate is greater than or equal to the overlap rate threshold and less than 1, the time series confidence is obtained by normalization calculation based on the difference between the spatiotemporal overlap rate and the overlap rate threshold.

[0079] When the spatiotemporal overlap rate is equal to 1, the time series confidence level is set to 1.

[0080] Furthermore, 10. The step of weighting the continuity anomaly, the brightness anomaly, the width anomaly, and the temporal confidence to obtain the comprehensive collapse confidence includes:

[0081] Each of the continuity anomaly, brightness anomaly, width anomaly, and temporal confidence is assigned a corresponding weighting coefficient; wherein the sum of each weighting coefficient is 1.

[0082] Based on the weighting coefficients, the continuity anomaly, the brightness anomaly, the width anomaly, and the temporal confidence are weighted and summed to obtain the overall collapse confidence.

[0083] The detection results include the spatial coordinates of the abnormal area, the type of abnormality, and the overall collapse confidence level.

[0084] The spatial location coordinates are obtained based on a pre-defined mapping relationship between image pixel coordinates and road surface physical coordinates;

[0085] The anomaly type is determined based on an anomaly index greater than zero; when only the continuity anomaly index is greater than zero, the anomaly type is a broken or missing line; when only the brightness anomaly index is greater than zero, the anomaly type is a brightness anomaly; when only the width anomaly index is greater than zero, the anomaly type is a width defect; when two or more anomaly indices are greater than zero, the anomaly type is a composite collapse.

[0086] This application introduces self-luminescent materials into road markings, enabling the markings to absorb light energy during the day and continuously emit light at night. When road collapses or structural damage occurs, the markings become damaged or missing, exhibiting detectable abnormal changes in luminescence continuity, brightness distribution, and width. Based on images acquired by a camera device, the integrity of the markings is quantitatively analyzed from three dimensions: continuity, brightness, and width. Multi-frame temporal consistency verification is combined to filter temporary interference such as vehicle obstruction. Finally, a comprehensive collapse confidence score is obtained through weighted fusion. This method requires no additional sensor equipment and utilizes the markings' own luminescent characteristics to achieve high-contrast imaging in low-light environments at night, effectively improving nighttime detection accuracy and system robustness, and realizing low-cost passive detection of road collapses. Attached Figure Description

[0087] Figure 1 This is a schematic diagram of the road collapse detection system provided in the embodiments of this application;

[0088] Figure 2 A schematic flowchart illustrating the road surface collapse identification method based on self-luminous coating provided in this application embodiment;

[0089] Figure 3 The images show the application effect of the self-luminous coating provided in this application at different times and the effect of road collapse. Detailed Implementation

[0090] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0091] Reference Figure 1This invention provides a road surface collapse detection system, which mainly includes a camera device 101 and a processing module 102. The camera device 101 is disposed on one side of the road surface, and the road surface contains a marking coating of self-luminous material. In this embodiment, the self-luminous material is preferably strontium aluminate self-luminous material.

[0092] The road marking coating in this case is applied to the road surface and contains strontium aluminate self-luminescent material. Specifically, the marking coating consists of strontium aluminate self-luminescent material, film-forming base material, pigments, fillers, and additives. The strontium aluminate self-luminescent material uses rare-earth-doped strontium aluminate long-afterglow luminescent powder, preferably SrAl2O4:Eu²⁺,Dy³⁺. The film-forming base material is an aqueous resin system, including but not limited to epoxy resin, fullerene-modified resin, etc., preferably silicone-acrylic emulsion or acrylic emulsion. Pigments and fillers include titanium dioxide and inorganic fillers. Additives include film-forming aids, dispersants, and defoamers. In the embodiments, the mass ratio of each component is: luminescent powder 35%–45%, preferably 40%; titanium dioxide 4%–8%, preferably 6%; calcium carbonate 5%–10%, preferably 8%; film-forming aid 1%–3%, preferably 2%; the remainder is film-forming base material and conventional additives. To improve the water resistance and stability of strontium aluminate long afterglow luminescent powder, it is preferable to perform surface coating treatment before coating preparation. The coating material can be silicate or organic polymer material. By forming a dense protective layer on the surface of the luminescent powder, the hydrolysis reaction in the water environment is inhibited, thereby improving the stability of its luminescent performance.

[0093] The preparation method of the road marking coating includes the following steps: First, strontium aluminate long-afterglow luminescent powder and an appropriate amount of dispersant are added to a solvent or emulsion system and uniformly dispersed by mechanical stirring or ultrasonic dispersion to prevent the luminescent powder from agglomerating; second, titanium dioxide and calcium carbonate and other pigments and fillers are added to the system and stirred to fully mix them to form a uniform slurry; then, film-forming base material and film-forming aid are added and stirred thoroughly to form a stable and uniform coating system; if necessary, defoamers and leveling agents are added to improve the coating's workability and surface quality; finally, the prepared coating is applied to the surface of the road substrate and formed a luminescent coating by natural drying or low-temperature curing.

[0094] The marking coating absorbs sunlight or ambient light during the day and continuously releases the stored light energy at night, thus forming luminous markings with long afterglow characteristics. It is particularly noteworthy that during nighttime driving, the light generated by vehicle headlights can also be reabsorbed by the strontium aluminate self-luminescent material, dynamically replenishing the stored energy. This characteristic is significant for extending the effective operating time of the detection system: even in the late night, due to the intermittent illumination provided by continuous traffic flow, the luminous intensity of the markings can be maintained above the level required for imaging by the camera device, thereby ensuring the reliability of detection throughout the night, extending its luminous duration, and enhancing its luminous intensity.

[0095] When road surface collapses or suffers structural damage, the marking coating will undergo changes such as cracking or fracturing, localized peeling, subsidence, or misalignment. These structural changes directly lead to physical damage to the luminescent layer, resulting in abnormal characteristics such as interrupted light emission, reduced local brightness, and irregular dark areas. Therefore, road surface collapses and related structural anomalies can be effectively characterized by changes in the luminescence state of the markings. It should be noted that this detection method requires no additional sensors; passive detection of road structural anomalies can be achieved simply by observing changes in the luminescence state of the markings themselves, significantly reducing system deployment costs.

[0096] A camera device 101 is installed on one side of the road surface, establishing a monitoring relationship with the road marking coating. The camera device 101 continuously monitors the road marking area, collecting image data of the area. In nighttime environments, due to the self-illuminating properties of the road marking coating, it can form high-brightness, high-contrast visual features in low-light conditions, effectively improving the imaging quality of the camera device 101 and enhancing the reliability of image processing and defect identification. This overcomes the strong dependence of traditional video detection methods on ambient light.

[0097] The processing module 102 is communicatively connected to the camera device 101, and is used to receive image data collected by the camera device 101, and execute the following road collapse recognition method based on self-luminous paint, analyze and judge the integrity of the road markings, and output the detection results.

[0098] Reference Figure 3To verify the application effect of self-luminous coating at different times, this invention generated application effect diagrams of the self-luminous coating based on Twinmotion simulation software. As shown in the diagrams, under daytime conditions, the self-luminous coating does not exhibit obvious luminescence characteristics, and its overall appearance is visually indistinguishable from ordinary road markings. At 6:00 PM, as the sun sets and ambient light gradually weakens, the self-luminous coating begins to release the light energy stored during the day, and the road markings gradually exhibit a brighter luminescence effect, significantly improving the visibility of the markings. As time progresses, the light energy stored within the material is gradually consumed, and its luminescence intensity shows a decreasing trend. At 10:00 PM, the marking brightness drops to a relatively low level, but its outline and spatial position are still clearly discernible, indicating that it still has good visibility performance at night. Furthermore, the diagrams also show the performance of the markings when the road surface collapses at night. It can be observed that the markings corresponding to the collapsed area show obvious anomalies; their luminescence continuity is disrupted, and they exhibit dark areas with irregular edges. This phenomenon further verifies the effectiveness and feasibility of self-luminous markings in detecting road structure anomalies.

[0099] Reference Figure 2 Based on the aforementioned road surface collapse detection system, this invention provides a road surface collapse identification method based on self-luminous coating, which includes the following steps:

[0100] S201. Obtain images of the road marking area using a camera device.

[0101] In this step, a camera device 101 installed on one side of the road surface continuously acquires images of the road marking area to obtain real-time road condition information. Nighttime is the primary application scenario for this invention. In nighttime environments, natural light intensity is low, and images acquired by traditional camera devices have poor signal-to-noise ratios and are easily affected by noise. However, because the marking coating contains strontium aluminate self-luminescent material, it can continuously release stored light energy at night, producing stable and continuous fluorescence emission, thereby significantly improving the visibility and contrast of the markings in low-light environments and providing high-quality image data for subsequent image processing and defect identification.

[0102] S202. Preprocess the image to obtain a preprocessed image.

[0103] In this step, basic processing is performed on the raw image captured by the camera device 101 to clearly distinguish the marked area from the background area. Step S202 specifically includes the following sub-steps:

[0104] S2021. Perform grayscale processing on the image to obtain a grayscale image.

[0105] Specifically, the original color image is converted to grayscale using the ITU-R BT.601 standard weighting method to reduce the dimensionality of the image data and focus on the brightness characteristics of the self-illuminating markings. The grayscale conversion formula is as follows:

[0106]

[0107] in, represents the pixel grayscale value of the converted image; R, G, and B are the red, green, and blue channel pixel components of the corresponding pixels in the original color image, respectively.

[0108] S2022. Based on the preset pixel range of the road surface area, the region of interest (ROI) is extracted from the grayscale image to obtain the ROI image.

[0109] Specifically, by combining the camera installation angle and fixed imaging rules of road surface photography, the row / column pixel range of the road surface area in the image is calibrated once during the system deployment phase, and a fixed binary mask of the road surface area is generated. During the algorithm operation, the mask remains fixed, and the gray values ​​of irrelevant areas such as the sky and road shoulders outside the mask are set to zero. Only the effective road surface area within the mask is retained for subsequent processing, thereby eliminating irrelevant background interference and reducing the computational load of subsequent algorithms.

[0110] S2023. Perform filtering and denoising processing on the ROI region image to obtain a denoised image.

[0111] In scheme A, Gaussian filtering is used for noise reduction to suppress Gaussian white noise and sensor particle noise commonly found in nighttime images. The formula for calculating the Gaussian filter kernel is:

[0112]

[0113] Where H(x,y) is the Gaussian filter kernel weight value at coordinates (x,y); x and y are the pixel relative coordinates with the center of the filter kernel as the origin; σ is the standard deviation of the Gaussian distribution, which controls the smoothness of the filter. In this scheme, σ=1.0.

[0114] In Scheme B, bilateral filtering is used instead of Gaussian filtering for noise reduction. This method utilizes both spatial neighborhood information and gray-level similarity of pixels to construct weights, effectively suppressing background particle noise and weakening halo diffusion interference while strictly preserving the edge contours of self-illuminating markers. The core weight calculation formula is as follows:

[0115]

[0116] in, The weights are the bilateral filtering weights for pixel (x,y) and its neighboring pixels (i,j); this scheme uses a 7×7 filtering kernel. The standard deviation of the spatial domain controls the influence range of the spatial neighborhood, with a value range of 5 to 8, and a preferred value of 7. The standard deviation of the grayscale domain controls the degree of influence of grayscale similarity, with a value range of 20 to 60, preferably 30; I(x,y) and I(i,j) are the grayscale values ​​of the corresponding pixels.

[0117] S2024. Perform adaptive histogram equalization on the denoised image to obtain a preprocessed image.

[0118] Specifically, the brightness of the denoised image is enhanced using the Adaptive Histogram Equalization (CLAHE) method. This method suppresses excessive noise amplification by limiting the upper limit of contrast, while locally enhancing image contrast, significantly improving the grayscale difference between the self-illuminating markings and the dark background road surface. Its pixel transformation formula is as follows:

[0119]

[0120] in, To output pixel grayscale values, For input pixel grayscale values, The cumulative distribution function value of pixel grayscale. The minimum cumulative distribution value is H0×W0, which is the total number of pixels in the image (H0 is the number of pixels in the image height, and W0 is the number of pixels in the image width), L is the total number of gray levels, which takes the value of 256, and floor(·) is the floor function.

[0121] S203. Extract the marking area based on the preprocessed image.

[0122] In this step, based on the preprocessed image For extracting the marking region, this invention provides two equivalent extraction methods. In specific applications, only one of these methods needs to be selected to complete the extraction of the marking region, without the need to execute both methods simultaneously. Method A is suitable for deployment scenarios with limited computing resources or where pre-trained models are not required, while Method B is suitable for application scenarios with higher segmentation accuracy requirements and deep learning inference capabilities.

[0123] Solution A: Based on edge detection and line segment analysis

[0124] S2031, Preprocessing the image Perform Sobel edge enhancement processing to obtain an edge-enhanced image.

[0125] Specifically, the gray-level gradients in the horizontal and vertical directions of the image are calculated using a 3×3 Sobel convolution kernel, and then the overall edge-enhanced image is synthesized using the gradient magnitudes. The core calculation formula is as follows:

[0126]

[0127]

[0128]

[0129] in, This is the horizontal gradient map of the image. This is the vertical gradient map of the image. This is a grayscale image of the ROI region after CLAHE enhancement. * indicates a two-dimensional convolution operation. This represents the gradient magnitude at pixel (x,y) of the synthesized edge-enhanced image. After Sobel processing, the self-illuminating markings are transformed from pre-processed bright blocks into clear linear edge contours, and noise interference from the background road surface is further suppressed.

[0130] It should be noted that during the Sobel edge enhancement process, the horizontal gradient... and vertical gradient The intermediate results are retained for use in subsequent LSD line segment detection algorithms to calculate gradient magnitude and gradient direction angle. Therefore, the gradient magnitude and gradient direction angle mentioned in subsequent steps are based on the intermediate results of the Sobel operation. and The result is calculated, rather than performing a second gradient operation on the edge-enhanced image.

[0131] S2032. Based on the edge-enhanced image, the LSD line segment detection algorithm is used to extract line segments and obtain an initial set of line segments.

[0132] The specific process for LSD segment extraction is as follows:

[0133] First, the gradient magnitude map obtained based on Sobel edge enhancement. A gradient threshold TG is set to pre-filter pixels, where the gradient threshold TG is 15% to 30% of the maximum gradient magnitude in the image, preferably 20%, retaining only those that meet the criteria. Pixels with a gradient ≥ TG are filtered out to remove low-gradient pseudo-edge pixels caused by background noise. Then, based on the pixels retained after gradient thresholding, the gradient direction field of the image is constructed using the gradient magnitude and gradient direction angle of each retained pixel obtained by Sobel operation.

[0134] Subsequently, based on the pixel gradient magnitude and gradient direction angle obtained from the Sobel operation, the gradient direction field of the image is constructed. The gradient direction angle at pixel (x,y) is then used. The calculation formula is:

[0135]

[0136] Where Gx(x,y) is the horizontal gradient scalar value at pixel (x,y), and Gy(x,y) is the vertical gradient scalar value at pixel (x,y).

[0137] The formula for the gradient magnitude at pixel (x, y) is:

[0138]

[0139] Then, set the orientation angle similarity threshold. The value ranges from 5° to 15°, preferably 10°. Pixels whose gradient direction angle difference between adjacent pixels is less than or equal to the direction angle similarity threshold are aggregated into line segment support regions. This constraint ensures that the aggregated pixels belong to the same self-illuminating mark edge.

[0140] Next, a minimum bounding rectangle is fitted to the support region of each line segment. Based on the endpoints, extension direction, and length of the rectangle's major axis, the geometric parameters of the initial line segments are obtained, generating an initial set of line segments. Its definition is:

[0141]

[0142] in, Let n be the k-th line segment, and n be the total number of line segments. ) is the starting coordinate of the k-th line segment, ( Let be the coordinates of the endpoint of the k-th line segment. Let be the angle of inclination of the line segment, and lnk be the length of the line segment. The formula for calculating the angle of inclination of a line segment is:

[0143]

[0144] The formula for calculating the length of a line segment is:

[0145]

[0146] It should be noted that the LSD line segment extraction algorithm described in this solution naturally supports the detection of curved self-illuminating road markings: for gently curved road markings, relying on the core feature of local linear detection of the algorithm, the local linear area of ​​the curved marking is independently extracted into short straight line segments, and finally a set of short line segments with sequentially changing direction and continuous connection of position is obtained, which can completely represent the contour features of the curved marking.

[0147] S2033. Perform segment length threshold filtering and segment inclination angle range filtering on the initial segment set to obtain a preliminary effective set of line segments. .

[0148] Specifically, set a line segment length threshold. The value is between 10 and 30 pixels, preferably 20 pixels, and only those that meet the requirements are retained. For line segments; in special scenarios where road collapse leads to broken and scattered markings, if the line segment is located in the area where self-illuminating markings are laid and its direction is consistent with the markings, even if the length is slightly less than [the specified length], [the following applies]. The segments that are deemed valid are retained. Simultaneously, an inclination angle range θk∈[-80°, 80°] is set to filter out interfering segments that are not part of the road marking direction. This inclination angle range is designed according to road traffic sign and marking construction standards, and can completely cover the direction of all straight and gently curved markings, while also reserving a margin to accommodate the shooting angle deviation of nighttime vehicle-mounted cameras.

[0149] S2034. The Cohen-Sutherland line segment clipping algorithm is used to clip the ROI region of the initial valid line segment set to obtain the final valid line segment set.

[0150] Specifically, based on a preset ROI window for road surface detection, each line segment's two endpoints are encoded using a 4-bit binary region, defining the start point as Code1 and the end point as Code2. If the codes of both endpoints of a line segment satisfy Code1 & Code2 = 0 (where & represents a bitwise AND operation), it indicates that the line segment is completely within the ROI window or on the window boundary, and is directly determined as a valid marking and retained in its entirety. If Code1 & Code2 ≠ 0, it indicates that the line segment is entirely outside the window or crosses the window boundary. In this case, by solving for the intersection of the line segment and the ROI window boundary, the endpoints are successively replaced and invalid line segments outside the window are trimmed, retaining only the valid marking segments within the ROI. After trimming, if the line segment length is zero, the line segment is directly discarded. Finally, only valid marking line segments within the ROI area are retained, and each line segment contains core geometric parameters such as position, direction, and length.

[0151] Solution B: Based on semantic segmentation

[0152] S2035. Normalize the preprocessed image to obtain a normalized image.

[0153] Specifically, the preprocessed image pixel values ​​are normalized to the [0,1] range to eliminate differences in pixel value magnitudes and mitigate numerical deviations caused by nighttime illumination fluctuations. The normalization formula is as follows:

[0154]

[0155] in, These are the preprocessed image pixel values. The minimum pixel value of the image. The maximum pixel value of the image. This is the normalized image.

[0156] S2036. Input the normalized image into the pre-trained UNet semantic segmentation network to obtain the binarized segmentation mask image Mask.

[0157] In this embodiment, the semantic segmentation network preferably adopts the UNet lightweight semantic segmentation network, but it is not limited to this. Other semantic segmentation networks with encoder-decoder structures (such as DeepLab, SegNet, etc.) can also be used.

[0158] The UNet network is based on an encoder-decoder structure. It extracts the depth features of the markings by downsampling and restores the pixel position and spatial resolution by upsampling. Finally, it outputs a binary segmentation mask image with the same size as the input image. In the mask, a pixel value of 1 indicates that it belongs to the self-illuminating marking area, and a pixel value of 0 indicates that it belongs to the road background area, thus achieving accurate separation of the markings and the background.

[0159] The UNet network training method used in this invention is as follows:

[0160] The training dataset consists of self-illuminating road marking images captured from real nighttime scenes and simulated rendered images, totaling approximately 5,000 images. Pixel-level binary annotations were completed manually, with the road marking area representing the foreground and the road surface and background representing negative samples.

[0161] The network adopts the standard UNet structure. The encoder has 4 layers with 64, 128, 256 and 512 channels respectively. Multi-scale feature extraction is achieved through convolution and pooling. The decoder has a corresponding 4-layer upsampling structure. It fuses low-level spatial information and high-level semantic information through skip connections. The final output is a segmentation probability map with the same size as the input.

[0162] The training process uses a joint loss function combining cross-entropy loss and Dice loss, as shown in the formula:

[0163]

[0164] in, For joint losses, Cross-entropy loss is used to measure the difference between the predicted probability distribution and the true label distribution. The Dice loss is used to address the issue of pixel imbalance between foreground and background. , The weighting coefficient is taken in this scheme. .

[0165] Among them, Dice lost The calculation formula is:

[0166]

[0167] In the formula, P is the marked area predicted by the model, and T is the marked area of ​​the actual region. This loss directly optimizes the segmentation accuracy by measuring the overlap between the prediction and the actual region.

[0168] The optimizer is Adam, and the initial learning rate is... The batch size was set to 4–8, and the number of training iterations was 50–100. Data augmentation strategies such as random flipping, brightness perturbation, and noise injection were used to improve the model's generalization ability. After training, the model can accurately distinguish between self-illuminating marking areas and background areas under low-light conditions at night.

[0169] S2037. Perform morphological closing operations on the binarized segmented mask image Mask to obtain a regularized mask image. .

[0170] Specifically, a morphological closing operation is performed on the binary segmented mask image Mask using a preset structuring element SE. The closing operation first dilates and then erodes, which can effectively repair minor breaks in the segmented markings, eliminate isolated small noise and holes, and make the marking contours more continuous and regular.

[0171] S2038, Based on regularized mask images Determine the marking area.

[0172] In Scheme B, to further improve the detection sensitivity of abnormal luminescence in the marking lines, the following auxiliary discrimination step is also included (this step is an additional processing unique to Scheme B and is not involved in Scheme A): The K-means clustering algorithm is used to process the preprocessed image. Based on pixel brightness features, the pixels were clustered into two categories, and the average brightness of the marked area was obtained for each category. and the average brightness of the background road surface area ; Calculate the brightness difference When the brightness difference Less than the preset brightness difference judgment threshold At that time, it is determined that the marking line has abnormal luminescence. Among them, the brightness difference judgment threshold is... The brightness is adaptively set to 30%~40% of the global average brightness. Since the self-luminous markings under normal working conditions have a significantly higher luminous intensity than the road background, when the road surface collapses, settles, or suffers structural damage, the markings may become damaged, buried, or the luminous layer may fail, resulting in a significant reduction in the brightness difference between the marking area and the road background. Therefore, by calculating the difference between the average brightness of the two, the degree of abnormality in the marking's luminescence can be effectively characterized, thereby enabling sensitive identification of anomalies related to road collapse.

[0173] It should be noted that the K-means clustering judgment step described above is an auxiliary judgment method in Scheme B. Its judgment result does not directly participate in the comprehensive confidence weighting calculation in S206, nor is it assigned an independent weight. When an abnormal luminescence is detected, the system logically marks the area as a high-risk area of ​​concern, giving it priority in subsequent manual review or alarm output. However, this does not change the calculation formulas and weight allocation for continuity anomaly, brightness anomaly, width anomaly, and temporal confidence.

[0174] S204. Based on the extracted gradation regions, the integrity of the gradation is analyzed from the dimensions of continuity, brightness, and width to obtain the continuity anomaly degree. Brightness anomaly and width anomaly .

[0175] The specific process of analyzing the integrity of the markings from the continuity dimension to obtain the continuity anomaly degree is as follows:

[0176] During the initial system deployment, undamaged, normal images of road markings are acquired. These images serve as a baseline reference for subsequent detection, used to establish standard features for the marking structure.

[0177] Based on the normal image, a single-pixel centerline skeleton for the grading line is extracted using a morphological thinning algorithm. Specifically, the morphological thinning algorithm repeatedly strips boundary pixels from the binary image of the grading line region until the width at each location is reduced to a single pixel, thereby obtaining a centerline skeleton that can characterize the direction and spatial position of the grading line. This skeleton can adapt to the straight or curved direction of the grading line, eliminating the influence of grading line width variations on subsequent length calculations.

[0178] The skeleton along the centerline is divided into several continuous small segments by a preset fixed step size. The fixed step size ranges from 10 to 30 pixels, with 20 pixels being the preferred value. The selection of the fixed step size must balance detection sensitivity and computational efficiency: a step size that is too small will increase computational load and increase sensitivity to noise, while a step size that is too large may miss local defects.

[0179] Calculate the arc length of each of the continuous segments segment by segment to obtain the standard length sequence of the arc segments. The standard length sequence of the arc segment is then permanently stored. This sequence uses arc length instead of straight-line distance to measure the length of each segment, which can effectively eliminate the influence of perspective distortion and curvature on length calculation and ensure the accuracy of the reference data.

[0180] During subsequent detection, the same morphological thinning algorithm is used to extract the centerline skeleton of the current frame's marker, and the same arc path is segmented with the fixed step size. The actual arc length of each segment is calculated segment by segment. By adopting the same skeleton extraction method and segmentation strategy as the deployment phase, it is ensured that the spatial positions of each segment in the current frame correspond one-to-one with the corresponding segments in the standard length sequence, making subsequent segment-by-segment comparisons comparable.

[0181] The actual arc length of each segment The standard arc length of the corresponding segment in the standard arc length sequence The comparison process focuses only on the shortening of the road markings due to collapse-related factors such as defects or breaks, and does not consider the lengthening of road markings as an anomaly of collapse.

[0182] When the actual arc length Less than the corresponding standard arc length At that time, the length loss rate of that segment is obtained by using the ratio of the difference between the standard arc length and the actual arc length to the standard arc length. The calculation formula is:

[0183]

[0184] in, Let i be the standard arc length of the i-th segment. Let be the actual arc length of the i-th segment.

[0185] like Then directly order This is not considered an anomaly related to collapse.

[0186] when When the threshold is 0.2, the continuity of the segment is determined to be abnormal.

[0187] When the actual arc length Greater than or equal to the corresponding standard arc length At that time, the length loss rate of this segment Setting it to zero means it is not considered a collapse-related anomaly.

[0188] The length loss rate Segments that are greater than or equal to a preset continuity anomaly threshold are identified as local continuity anomaly segments.

[0189] The continuity anomaly threshold is preferably set to 0.2, that is, when the actual length of a certain section of the marking is shortened by more than 20% of the standard length, it is determined that there is a local continuity anomaly in that section.

[0190] The proportion of locally discontinuous anomalies in the total number of segments is statistically analyzed. This ratio is used as the continuity anomaly degree. :

[0191]

[0192] The value range is [0,1]. The larger the value, the more severe the local breakage, missing parts, or pothole damage of the road markings, and the higher the probability of road surface collapse.

[0193] The integrity of the markings was analyzed from the perspective of brightness to obtain the brightness anomaly level. The specific process is as follows:

[0194] Calculate the overall average brightness of the marked area The formula is:

[0195]

[0196] in, Let I(x,y) be the total number of pixels in the marked area, and let I(x,y) be the grayscale value at pixel (x,y). The overall average brightness serves as a global reference benchmark for the brightness levels of subsequent local sub-regions.

[0197] A sliding window of a preset size traverses the marked area row by row and column by column, with each area covered by the sliding window considered a local sub-region. The size of the sliding window ranges from 5×5 to 21×21 pixels, preferably 11×11 pixels. The sliding step size is half the size of the sliding window to ensure overlapping coverage between adjacent sub-regions and avoid missing local dark areas. The selection of the sliding window size must balance detection granularity and coverage integrity: a window that is too small is easily affected by single-pixel noise, while a window that is too large may mix abnormal areas with normal areas, reducing detection sensitivity.

[0198] For each of the aforementioned local sub-regions, calculate the average brightness of each local sub-region. Specifically, the sum of the grayscale values ​​of all pixels within a local sub-region is divided by the total number of pixels in that sub-region. Obtain the average brightness of this sub-region. .

[0199] Based on the overall average brightness and the total number of pixels in the local sub-region Calculate the average brightness of the caliper line after removing the current local sub-region, and use it as the reference brightness value for that local sub-region. The calculation formula is:

[0200]

[0201] in, This represents the total number of pixels within the current local sub-region. By removing the brightness contribution of the current sub-region itself when calculating the reference value, the interference of low brightness in abnormal sub-regions on their own reference benchmark can be avoided, making the brightness comparison more objective and accurate.

[0202] Based on the average brightness of the local sub-region and the corresponding reference brightness value The ratio is used to obtain the local brightness anomaly of each of the local sub-regions. The calculation formula is:

[0203]

[0204] Local brightness anomaly The larger the value, the lower the brightness of the sub-region relative to other normal areas of the caliper, and the more pronounced the dark area characteristics.

[0205] The local brightness anomaly Local sub-regions with brightness levels greater than or equal to a preset brightness anomaly threshold are identified as brightness anomaly regions. The preferred brightness anomaly threshold is 0.5, meaning that when the brightness of a local sub-region is more than 50% lower than a reference brightness value, the region is considered to have a brightness anomaly.

[0206] The proportion of the brightness anomaly regions to the total number of local sub-regions is statistically analyzed. This ratio is used as the brightness anomaly. ,Right now:

[0207]

[0208] Brightness anomaly The value range is [0,1]. The larger the value, the more severe the local darkening of the road markings due to collapse, subsidence, or burial.

[0209] The integrity of the gradation line is analyzed from the width dimension to obtain the width anomaly degree. The specific process is as follows:

[0210] During the initial system deployment, undamaged, normal images of road markings are acquired. These images are used to establish a baseline reference for the marking width.

[0211] Based on the normal image, the pixel width of the markers within the region of interest is counted line by line, and the average width of each line of multiple consecutive frames of the normal image is taken to obtain a sequence of standard values ​​for line widths. The standard value sequence of row widths is then permanently stored. By averaging across multiple frames, random jitter errors in a single frame image can be eliminated, improving the stability of the benchmark data. This sequence records the standard width of the datum line row by row, naturally incorporating the "nearer is larger, farther is smaller" width variation caused by image perspective distortion, and can be directly used as a comparison benchmark for each row.

[0212] During subsequent detection, the actual width of the marking line is calculated line by line in the current frame image. Specifically, the number of consecutive pixels belonging to the gradation area in each row is counted, and this number is used as the actual width of that row.

[0213] The actual width of each row The standard width value of the corresponding row in the row-level width standard value sequence A comparison is made. This comparison process focuses only on cases where the width of the road markings has narrowed due to factors related to collapse.

[0214] When the actual width Smaller than the corresponding standard width value At that time, the width deviation rate of the row is obtained by the ratio of the difference between the standard width value and the actual width to the standard width value. The calculation formula is:

[0215]

[0216] in, This is the standard width value for the i-th row. This represents the actual width of the i-th row.

[0217] like (If the markings do not narrow or even widen), then directly... This is not considered an anomaly related to collapse.

[0218] when When the actual width of a line is reduced by more than 30% of the standard width, the line width is considered abnormal.

[0219] Calculate the proportion of rows with abnormal widths within the ROI out of the total number of rows. Then the width anomaly is:

[0220]

[0221] The value range is [0,1]. The larger the value, the more severe the stretching, breaking, and thinning of the road markings due to collapse.

[0222] The system supports periodic updates of the standard length sequence and row width standard value sequence to adapt to natural wear and luminescence decay during long-term use of the markings, ensuring detection stability.

[0223] S205. Perform temporal consistency verification on the abnormal regions detected in multiple consecutive frames of images to obtain temporal confidence. .

[0224] This step serves as a crucial verification step after the pavement marking integrity assessment, distinguishing between actual road surface collapses and temporary interference. This step analyzes multiple consecutive frames of the aforementioned images. In practical applications, vehicle traffic can cause momentary occlusion of pavement markings, easily interfering with the detection results. To reduce the false positive rate, this step analyzes consecutive frames of the aforementioned images. The frame image is used for anomaly region matching and spatiotemporal overlap rate calculation, where... The value range is 3 to 5 frames, with a preferred value of 3 frames; let the abnormal region detected in the t-th frame be... ,but Spatiotemporal overlap rate of anomalous regions in consecutive frames of images Defined as:

[0225]

[0226] Wherein, the numerator is the continuous numerator starting from frame t (the starting frame). The pixel area of ​​the intersection of abnormal regions in the frame, with the denominator being the abnormal region in frame t (the starting frame). The formula measures the pixel area. This formula is used to measure whether an abnormal region detected in the initial frame persists stably in subsequent consecutive frames.

[0227] The spatiotemporal overlap rate overlap rate threshold with preset Compare the results and calculate the time series confidence level. Among them, the overlap rate threshold The preferred value is 0.8.

[0228] When the spatiotemporal overlap rate Less than the overlap rate threshold At that time, the time series confidence level Set to zero to determine that the abnormal area is a temporary interference such as vehicle obstruction and remove it;

[0229] When the spatiotemporal overlap rate Greater than or equal to the overlap rate threshold And when it is less than 1, according to the spatiotemporal overlap rate With the overlap rate threshold The difference is normalized and the time series confidence score is calculated. The formula is:

[0230]

[0231] in, The overlap rate threshold is preferably set to 0.8. When the value is greater than 0, the abnormal area is determined to exist stably in multiple consecutive frames of images, indicating a real road surface collapse. The value is [0,1] when When the time series confidence level is 0, it is considered a temporary disturbance and is removed. The higher the time series confidence level, the higher the credibility that the abnormal area represents a real road surface collapse.

[0232] Furthermore, to more effectively distinguish between road surface collapse and vehicle interference, this invention also employs a multi-dimensional comprehensive discrimination strategy, as follows:

[0233] In terms of time, vehicle occlusion is usually short-lived and random, while road collapse is persistent. By comparing the temporal sequence of continuously acquired images, if an abnormal area persists in multiple frames with minimal changes, it is determined to be an abnormal road structure; conversely, if the abnormality only appears briefly in a few frames, it is more likely to be caused by vehicle occlusion.

[0234] In terms of spatial location, the vehicle's position changes continuously over time, while the collapsed area remains relatively fixed in space. By registering and comparing multiple frames of images, it is determined whether the position of the abnormal area remains stable; if its spatial position remains basically unchanged, it is likely to be judged as an abnormal road structure; otherwise, it can be regarded as dynamic occlusion interference.

[0235] In terms of morphological characteristics, vehicles typically have relatively regular geometric contours, while road collapse areas often exhibit irregular shapes. By analyzing the boundary shape, curvature changes, and regional compactness of abnormal areas, the two types of targets can be effectively distinguished.

[0236] Regarding the correlation between road markings and their structure, road subsidence often directly affects the marking structure, manifesting as breakage, misalignment, or partial loss of the markings. Abnormal areas and changes in the marking structure exhibit strong spatial consistency. In contrast, vehicle obstruction only affects the visibility of the markings and does not alter their actual structure. Analyzing the correlation between abnormal areas and the marking structure can further improve the accuracy of identification.

[0237] By fusing and analyzing the above multi-dimensional information, interference caused by vehicle obstruction can be effectively suppressed, significantly reducing the probability of false detection and misjudgment, thereby improving the detection accuracy and robustness of the system in complex traffic environments.

[0238] S206. The comprehensive collapse confidence P is obtained by weighting the continuity anomaly, brightness anomaly, width anomaly and temporal confidence.

[0239] The overall collapse confidence level P is calculated using the following formula:

[0240]

[0241] in, , , , These are the weighting coefficients for each indicator. For continuity anomaly degree, For brightness anomaly, For width anomaly, This represents the time series confidence level. The weighting coefficients satisfy the normalization constraint. + + + = 1. Weights are allocated based on the contribution of each dimension to the collapse detection: temporal consistency is the core of authenticity determination and has the highest weight; continuity and brightness are direct characteristics of collapse and have the next highest weight; width anomalies are auxiliary features and have a relatively low weight. Each weight coefficient can be obtained through statistical analysis of historical detection samples, experimental calibration, or online adaptive adjustment; in one embodiment, the preferred weight values ​​are: = 0.25, = 0.25, = 0.2, = 0.3. The closer the overall collapse confidence score is to 1, the higher the confidence level that the area represents a genuine road surface collapse.

[0242] S207. When the overall collapse confidence level P is greater than or equal to the preset collapse judgment threshold P final When the system detects a road surface collapse in the corresponding area, it outputs the detection results.

[0243] In this step, if the overall collapse confidence level P ≥ P final (If the final collapse judgment threshold is set to 0.7, it is judged as a road collapse and a warning is output. The detection results include the spatial coordinates of the abnormal area, the anomaly type, and the comprehensive collapse confidence level P.)

[0244] Specifically, the results of the three criteria—continuity, brightness, and width—are first logically merged to obtain anomaly candidate masks in a single frame image; then, temporal consistency verification is performed to check the continuity... The spatiotemporal overlap rate of the anomaly candidate mask in the frame is calculated to filter out unstable interference regions, thus obtaining the final true anomaly region. Contour detection is then performed on the true anomaly region, its complete contour point set is extracted, and a minimum bounding rectangle is fitted to the contour point set. The rectangular frame can completely represent the image pixel boundaries of road surface collapse or missing markings. During the deployment phase, the system completes calibration by setting road surface marker points, solves the homography matrix based on perspective transformation, and establishes a mapping relationship between image pixel coordinates and local planar physical coordinates of the road surface (unit: meters), so as to achieve accurate output of the actual relative position of abnormal areas.

[0245] Simultaneously, the geometric center coordinates are calculated based on the vertices of the rectangle:

[0246]

[0247] It is then converted into local physical coordinates of the road surface through a pre-defined mapping relationship, serving as a rapid positioning reference for abnormal areas.

[0248] The anomaly type is determined based on whether each anomaly degree exceeds the corresponding anomaly percentage threshold: the continuity anomaly percentage threshold is preferably 0.1, the brightness anomaly percentage threshold is preferably 0.1, and the width anomaly percentage threshold is preferably 0.1. When only the continuity anomaly degree is greater than or equal to the corresponding anomaly percentage threshold, it is determined to be a broken or missing marking; when only the brightness anomaly degree is greater than or equal to the corresponding anomaly percentage threshold, it is determined to be a brightness anomaly; when only the width anomaly degree is greater than or equal to the corresponding anomaly percentage threshold, it is determined to be a width defect; when two or more anomalies are greater than or equal to the corresponding anomaly percentage threshold, it is determined to be a composite collapse. If it is a curved anomaly area, the gradual change pattern of the marking direction is considered to determine whether it is an arc deformation of the marking caused by local road surface settlement.

[0249] S208, Daytime Detection Mode.

[0250] It should be noted that the above-mentioned S201 to S207 are mainly used for nighttime detection. Under daytime conditions, due to the high intensity of ambient light, the luminous effect of self-illuminating markings is not obvious and cannot be used as the main criterion. Therefore, detection methods based on appearance features can be used as a supplement to achieve all-weather road condition monitoring.

[0251] Specifically, road surface images are acquired using camera devices and preprocessed with grayscale conversion, ROI extraction, and filtering consistent with nighttime data. Marking regions are extracted based on the fusion of multiple features including color, edge, and texture, and their geometric structural features such as location, width, direction, and boundary morphology are analyzed. Building upon this, the continuity, width anomaly, and temporal confidence calculation logic used in nighttime detection is applied to determine if markings exhibit anomalies such as breaks, offsets, missing parts, or local distortions. When significant deformation or continuity disruption is detected in the markings, a comprehensive judgment is made based on spatial distribution and the degree of anomaly. If abnormal subsidence, irregular boundary changes, or overall structural shifts occur in a localized area of ​​the marking, the corresponding area is determined to have road surface collapse or structural anomalies.

[0252] The daytime detection method and the nighttime detection method use the same algorithm framework. Only the marking extraction method switches adaptively with the light, while the rest of the process is completely the same. By combining the nighttime luminous mode with the daytime morphological mode, the system can maintain stable detection capability under different lighting conditions and achieve continuous monitoring of road surface conditions around the clock.

[0253] In summary, the overall operation flow of the system of this invention is as follows: During the day, the self-luminous road markings absorb solar radiation and ambient light energy to complete energy storage, and can also monitor the road markings through a daytime morphological detection mode; at night, the markings continuously release the stored light energy, forming stable and clear visual features, and can also absorb external light sources such as vehicle headlights to achieve dynamic energy replenishment; the camera device continuously acquires images of the road marking area; the system processes the acquired images, extracts the marking area, and comprehensively analyzes its structural integrity, continuity, and brightness distribution characteristics; when an anomaly is detected in the markings, it is identified and judged by combining spatial, temporal, and morphological features; the confirmed road collapse area is located and a warning message is output. Through the above process, this invention can realize road collapse detection based on self-luminous coating, with all-weather operation capability and high reliability.

[0254] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for identifying road surface collapse based on self-luminous coating, characterized in that, An application is made to a road subsidence detection system, which includes a camera device and a processing module mounted on one side of the road surface. The road surface contains a marking coating made of self-luminous material, which absorbs light energy during the day and continuously emits light at night. The method includes: The camera device acquires images of the road marking area; The image is preprocessed to obtain a preprocessed image; Based on the preprocessed image, extract the marking region; Based on the extracted marking regions, the integrity of the markings is analyzed from the dimensions of continuity, brightness, and width, respectively, to obtain the continuity anomaly, brightness anomaly, and width anomaly. Temporal consistency verification is performed on abnormal regions detected in multiple consecutive frames of the image to obtain temporal confidence. The comprehensive collapse confidence is obtained by weighting the continuity anomaly, the brightness anomaly, the width anomaly, and the temporal confidence. When the overall collapse confidence level is greater than or equal to the preset collapse judgment threshold, it is determined that there is a road collapse in the corresponding area, and the detection result is output.

2. The method according to claim 1, characterized in that, The preprocessing of the image to obtain a preprocessed image includes: The image is converted to grayscale to obtain a grayscale image; Based on a preset pixel range for the road surface area, the region of interest (ROI) is extracted from the grayscale image to obtain an ROI image. The ROI region image is filtered and denoised to obtain a denoised image; The denoised image is subjected to adaptive histogram equalization to obtain the preprocessed image.

3. The method according to claim 2, characterized in that, The step of filtering and denoising the ROI region image to obtain a denoised image includes: The ROI region image is denoised using a Gaussian filtering method. The Gaussian filtering method is based on a Gaussian distribution function to construct a filter kernel, and performs a weighted average operation on each pixel and its neighboring pixels to obtain the denoised image.

4. The method according to claim 2, characterized in that, The step of filtering and denoising the ROI region image to obtain a denoised image includes: The ROI region image is denoised using a bilateral filtering method. The bilateral filtering method constructs weights based on the spatial distance and gray-level similarity between pixels, and performs a weighted average operation on each pixel and its neighboring pixels to obtain the denoised image.

5. The method according to claim 2, characterized in that, The step of extracting the marking region based on the preprocessed image includes: The preprocessed image is then subjected to Sobel edge enhancement processing to obtain an edge-enhanced image; Based on the edge-enhanced image, line segments are extracted using the LSD line segment detection algorithm to obtain an initial set of line segments; The initial set of line segments is filtered by line segment length threshold and line segment tilt angle range to obtain a preliminary set of effective line segments; The Cohen-Sutherland line segment clipping algorithm is used to clip the ROI region of the preliminary set of valid line segments to obtain the final set of valid line segments. The marking area is determined based on the final set of valid marking segments.

6. The method according to claim 5, characterized in that, Based on the edge-enhanced image, line segments are extracted using the LSD line segment detection algorithm to obtain an initial set of line segments, including: Based on the grayscale gradients of each pixel in the horizontal and vertical directions obtained during the Sobel edge enhancement process, the gradient magnitude and gradient direction angle of each pixel are calculated. Set a gradient threshold, retain pixels with gradient magnitude greater than or equal to the gradient threshold, and filter background noise pixels with gradient magnitude lower than the gradient threshold; Wherein, the gradient threshold is a preset percentage of the maximum gradient magnitude in the edge enhancement image; Based on the retained pixels, a direction angle similarity threshold is set, and adjacent pixels among the retained pixels whose gradient direction angle difference is less than or equal to the direction angle similarity threshold are aggregated into line segment support regions. The minimum bounding rectangle is fitted to the support region of each line segment to obtain the starting point coordinates, ending point coordinates, inclination angle and length of each line segment; The initial set of line segments is generated based on the starting point coordinates, the ending point coordinates, the inclination angle, and the length of each line segment.

7. The method according to claim 2, characterized in that, The step of extracting the marking region based on the preprocessed image includes: The preprocessed image is normalized to obtain a normalized image; The normalized image is input into a pre-trained semantic segmentation network to obtain a binarized segmentation mask image, wherein the region with a pixel value of 1 in the binarized segmentation mask image represents the marking region, and the region with a pixel value of 0 represents the road background region. Perform morphological closing operation on the binarized segmented mask image to obtain a regularized mask image; The morphological closing operation is a composite operation that sequentially performs dilation and erosion operations; The marking region is determined based on the regularized mask image.

8. The method according to claim 1, characterized in that, The analysis of the integrity of the markings from the continuity dimension to obtain the continuity anomaly degree includes: When the system is first deployed, normal images of undamaged road markings are collected; Based on the normal image, the single-pixel centerline skeleton of the marking is extracted using a morphological thinning algorithm; The skeleton along the centerline is divided into several continuous small segments by a preset fixed step size. The arc length of each of the continuous small segments is calculated segment by segment to obtain the standard length sequence of the arc segment, and the standard length sequence of the arc segment is stored in a fixed manner. During subsequent detection, the same morphological thinning algorithm is used to extract the centerline skeleton of the current frame line, and it is segmented along the same arc path with the fixed step size, and the actual arc length of each segment is calculated segment by segment; The actual arc length of each segment is compared with the standard arc length of the corresponding segment in the standard arc length sequence; When the actual arc length is less than the corresponding standard arc length, the length loss rate of that segment is obtained by the ratio of the difference between the standard arc length and the actual arc length to the standard arc length. When the actual arc length is greater than or equal to the corresponding standard arc length, the length loss rate of that segment is set to zero. Segments with a length loss rate greater than or equal to a preset continuity anomaly threshold are identified as local continuity anomaly segments. The proportion of the locally discontinuous segments to the total number of segments is calculated, and this proportion is used as the degree of discontinuity.

9. The method according to claim 1, characterized in that, The analysis of the integrity of the markings from the brightness dimension to obtain the brightness anomaly degree includes: Calculate the overall average brightness of the marked area; The sliding window of a preset size is used to traverse the marked area row by row and column by column, and the area covered by each sliding window is taken as a local sub-region. For each of the aforementioned local sub-regions, calculate the average brightness of each local sub-region; Based on the overall average brightness and the total number of pixels in the local sub-region, calculate the average brightness of the caliper after removing the current local sub-region, and use it as the reference brightness value of the local sub-region. The local brightness anomaly degree of each local sub-region is obtained based on the ratio of the average brightness of the local sub-region to the corresponding reference brightness value; Local sub-regions with a local brightness anomaly level greater than or equal to a preset brightness anomaly threshold are identified as brightness anomaly regions. The proportion of the brightness anomaly region to the total number of local sub-regions is calculated, and this proportion is taken as the brightness anomaly degree. The analysis of the integrity of the grading line from the width dimension to obtain the width anomaly degree includes: When the system is first deployed, normal images of undamaged road markings are collected; Based on the normal image, the pixel width of the markings in the region of interest is counted line by line, and the average width of each line of the normal image in multiple consecutive frames is taken to obtain a row-level width standard value sequence, and the row-level width standard value sequence is stored in a fixed manner. During subsequent detection, the actual width of the marking line is calculated line by line for the current frame image; The actual width of each row is compared with the standard width value of the corresponding row in the row-level width standard value sequence; When the actual width is less than the corresponding standard width value, the width deviation rate of the row is obtained by the ratio of the difference between the standard width value and the actual width to the standard width value. When the actual width is greater than or equal to the corresponding standard width value, the width deviation rate of the row is set to zero; Rows with a width deviation rate greater than or equal to a preset width anomaly threshold are identified as width anomaly rows; The proportion of rows with abnormal widths to the total number of rows is calculated, and this proportion is used as the width abnormality degree. The step of performing temporal consistency verification on abnormal regions detected in multiple consecutive frames of images to obtain temporal confidence includes: Obtain the abnormal regions detected in each frame of the image across multiple consecutive frames; Calculate the intersection pixel area of ​​the abnormal regions in each of the consecutive frames; The spatiotemporal overlap rate is obtained by the ratio of the area of ​​the intersection pixels to the area of ​​the pixel region of the abnormal region in the initial frame. The spatiotemporal overlap rate is compared with a preset overlap rate threshold. When the spatiotemporal overlap rate is less than the overlap rate threshold, the temporal confidence is set to zero. When the spatiotemporal overlap rate is greater than or equal to the overlap rate threshold and less than 1, the time series confidence is obtained by normalization calculation based on the difference between the spatiotemporal overlap rate and the overlap rate threshold. When the spatiotemporal overlap rate is equal to 1, the time series confidence level is set to 1.

10. The method according to claim 1, characterized in that, The step of weighting the continuity anomaly, brightness anomaly, width anomaly, and temporal confidence to obtain a comprehensive collapse confidence score includes: Each of the continuity anomaly, brightness anomaly, width anomaly, and temporal confidence is assigned a corresponding weighting coefficient; wherein the sum of each weighting coefficient is 1. Based on the weighting coefficients, the continuity anomaly, the brightness anomaly, the width anomaly, and the temporal confidence are weighted and summed to obtain the overall collapse confidence. The detection results include the spatial coordinates of the abnormal area, the type of abnormality, and the overall collapse confidence level. The spatial location coordinates are obtained based on a pre-defined mapping relationship between image pixel coordinates and road surface physical coordinates; The anomaly type is determined based on whether each anomaly degree exceeds the corresponding anomaly percentage threshold; when only the continuity anomaly degree is greater than or equal to the corresponding anomaly percentage threshold, the anomaly type is a broken or missing line; when only the brightness anomaly degree is greater than or equal to the corresponding anomaly percentage threshold, the anomaly type is a brightness anomaly; when only the width anomaly degree is greater than or equal to the corresponding anomaly percentage threshold, the anomaly type is a width defect; when two or more anomaly degree indicators are greater than or equal to the corresponding anomaly percentage threshold, the anomaly type is a composite collapse.

Images