Road pavement pedestrian crossing damage detection method and system

Abstract

The invention relates to the technical field of pavement damage detection, in particular to a highway pavement pedestrian crossing damage detection method and system, and the method comprises the steps: obtaining a high-resolution video image of a to-be-detected highway pavement, and extracting a plurality of frames of pedestrian crossing region images; the pedestrian crossing area image is input into a preset damage detection model, suspected damage areas and corresponding position coordinates are obtained, and the damage detection model is used for conducting feature extraction, damage judgment and damage area segmentation on the pedestrian crossing area image; judging the damage degree of the suspected damage area, and determining a real damage area according to the damage degree; and integrating the same real damaged area detection results corresponding to the multiple frames of pedestrian crossing area images to obtain a final damaged area, and matching the final damaged area with a high-precision map to obtain accurate geographic position information, thereby completing pedestrian crossing damage detection of the highway pavement to be detected. According to the invention, automatic detection and positioning of pedestrian crossing damage can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of road surface damage detection, and in particular to a method and system for detecting damage to a pedestrian crossing on a highway road surface. Background Art

[0002] There is a technical contradiction in the process of detecting damage to pedestrian crossings on highway pavements: in order to fully and accurately detect the damage to pedestrian crossings, it is necessary to collect high-resolution, high-frame-rate image data, but this will generate a huge amount of data, which will bring huge pressure to data transmission and storage. At the same time, in order to detect and analyze the damage in real time, image data needs to be processed quickly, but the processing speed of massive high-definition image data is difficult to meet the real-time requirements. In addition, pedestrian crossing damage detection needs to be carried out under various harsh external conditions, such as lighting changes, rainy and snowy weather, vehicle occlusion, etc. These factors will have an adverse effect on image quality and recognition accuracy.

[0003] How to reduce data transmission and storage pressure and improve the environmental adaptability of the system while ensuring detection accuracy and real-time performance is a key technical issue faced by the road pavement pedestrian crosswalk damage detection method and system. This requires technical innovation and optimization in multiple links such as image acquisition, data compression, transmission optimization, and algorithm design, balancing the contradictions between data quality, processing speed, and system stability, and developing efficient and reliable damage detection solutions to provide an important basis for road maintenance decisions. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for detecting damaged pedestrian crossings on highway pavements in response to the above-mentioned technical problems, so as to realize the automatic detection and positioning of damaged pedestrian crossings, provide accurate decision-making basis for road maintenance, and effectively improve road safety and maintenance efficiency.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A method for detecting damage to a pedestrian crossing on a highway pavement, comprising:

[0007] Obtain a high-resolution video image of the road surface to be inspected, and extract several frames of pedestrian crossing area images;

[0008] Inputting the pedestrian crossing area image into a preset damage detection model to obtain suspected damaged areas and corresponding position coordinates, wherein the damage detection model is used to perform feature extraction, damage judgment, and damaged area segmentation on the pedestrian crossing area image;

[0009] Determining the degree of damage of the suspected damaged area, and determining the actual damaged area according to the degree of damage;

[0010] The detection results of the same real damaged area corresponding to multiple frames of pedestrian crossing area images are integrated to obtain the final damaged area, and the final damaged area is matched with the high-precision map to obtain accurate geographic location information, thereby completing the pedestrian crossing damage detection on the road surface to be detected.

[0011] Optionally, the step of preprocessing the crosswalk area image before inputting the crosswalk area image into the predicted damage detection model includes:

[0012] Binarizing the crosswalk area image, performing shape matching on the binarized crosswalk area image, determining the precise position of the crosswalk area, and capturing a first crosswalk image in the high-resolution video image based on the precise position;

[0013] Performing image segmentation processing on the first crosswalk image to obtain a second crosswalk image;

[0014] Perform image enhancement and denoising processing on the second crosswalk image to obtain a third crosswalk image.

[0015] Optionally, the damage detection model includes: a feature extraction module, a damage detection module, and a region segmentation module, wherein the feature extraction module is used to extract features of the crosswalk area image based on a convolutional neural network to obtain image features; the damage detection module is used to classify the image features for damage based on a classifier to obtain damage detection results; the region segmentation module is used to locate and segment the damaged area when damage is detected, and output the outline and position coordinates of the damaged area.

[0016] Optionally, judging the damage degree of the suspected damaged area and determining the real damaged area according to the damage degree includes:

[0017] Dividing the suspected damaged area into image sub-blocks;

[0018] Extracting texture features and shape features of the image sub-blocks respectively, and constructing feature vectors;

[0019] Processing the feature vector by a support vector machine to obtain the damage degree of the suspected damaged area;

[0020] If the damage degree exceeds a preset damage threshold, the suspected damaged area is determined as a real damaged area; if the damage degree does not exceed the damage threshold, the suspected damaged area is determined as an undamaged area.

[0021] Optionally, integrating the same real damaged area detection results corresponding to multiple frames of pedestrian crossing area images includes:

[0022] Perform feature extraction on the same real damaged area detection results corresponding to multiple frames of pedestrian crossing area images to obtain multi-scale feature vectors;

[0023] The multi-scale feature vectors are fused, and damage identification and damage degree judgment are performed based on the fused feature vectors to obtain the final damaged area.

[0024] Optionally, matching the final damaged area with a high-precision map to obtain accurate geographic location information includes:

[0025] Mapping the location coordinates of the final damaged area to the high-precision map and marking the damaged points;

[0026] The damaged points are grouped by a clustering algorithm to identify the densely-damaged areas and obtain the precise geographic location information.

[0027] To further achieve the above-mentioned object, the present invention also provides a highway pavement pedestrian crosswalk damage detection system, comprising: a test image acquisition module, a damage area detection module, and a damage area integration module;

[0028] The image acquisition module to be tested is used to acquire a high-resolution video image of the road surface to be tested and extract a plurality of frames of images of the crosswalk area;

[0029] The damaged area detection module is used to input the pedestrian crossing area image into a preset damage detection model, obtain the suspected damaged area and the corresponding position coordinates, and determine the damage degree of the suspected damaged area, and determine the real damaged area according to the damage degree;

[0030] The damaged area integration module is used to integrate the detection results of the same real damaged area corresponding to multiple frames of pedestrian crossing area images to obtain the final damaged area, and match the final damaged area with the high-precision map to obtain accurate geographic location information to complete the pedestrian crossing damage detection on the highway road surface to be detected.

[0031] Optionally, the image acquisition module to be tested includes: a video segmentation unit and an image processing unit, wherein the video segmentation unit is used to segment and extract a plurality of frames of pedestrian crossing area images of different scales from the high-resolution video image of the road surface to be tested; and the image processing unit is used to perform binarization, image segmentation, image enhancement, and denoising on the pedestrian crossing area image to obtain the processed pedestrian crossing area image.

[0032] Optionally, the damaged area detection module includes: a damaged area detection unit and a damage degree judgment unit, wherein the damaged area detection unit is used to perform feature extraction, feature recognition, and area segmentation on the crosswalk area image based on the damage detection model, and output the contour and position coordinates of the suspected damaged area; the damage degree judgment unit is used to extract texture features and shape features of the suspected damaged area, judge the degree of damage based on the texture features and shape features, and determine the actual damaged area according to the degree of damage.

[0033] Optionally, the damaged area integration module includes: a multi-scale fusion unit and a high-precision positioning unit, wherein the multi-scale fusion unit is used to extract features from the same real damaged area detection results corresponding to multiple frames of pedestrian crossing area images, obtain multi-scale feature vectors and fuse them, and identify the final damaged area based on the fused feature vector; the high-precision positioning unit is used to map the location coordinates of the final damaged area to a high-precision map and mark the damaged points, group the damaged points through a clustering algorithm, identify densely damaged areas and obtain precise geographic location information.

[0034] The beneficial effects of the present invention are:

[0035] The present invention first extracts the crosswalk area from the high-resolution highway video, and performs image segmentation and enhancement processing; then uses a convolutional neural network to detect suspected damaged areas, and combines the support vector machine algorithm to classify the degree of damage. In order to improve the detection accuracy, the present invention adopts a multi-scale fusion strategy to integrate the detection results of different frames, and finally matches the damage information with the high-precision map to obtain the precise geographic location and upload it to the road condition management platform. The present invention realizes the automatic detection and positioning of crosswalk damage, provides an accurate decision-making basis for road maintenance, and effectively improves road safety and maintenance efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 A flow chart of a method for detecting damage to a pedestrian crosswalk on a highway pavement according to an embodiment of the present invention;

[0038] Figure 2 A flow chart of a method for preprocessing a crosswalk area image according to an embodiment of the present invention;

[0039] Figure 3This is a flow chart of a method for determining a real damaged area according to the damage degree according to an embodiment of the present invention;

[0040] Figure 4 A schematic diagram of the composition structure and workflow of a highway pavement pedestrian crosswalk damage detection system according to an embodiment of the present invention;

[0041] Figure 5 The structure and workflow diagram of the image acquisition module to be tested according to an embodiment of the present invention;

[0042] Figure 6 This is a diagram showing the structure and workflow of a damaged area detection module according to an embodiment of the present invention;

[0043] Figure 7 1 is a diagram showing the structure and workflow of the damaged area integration module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] Crosswalks are important passages for pedestrians. Damaged pavement may cause pedestrians to fall or other safety accidents. Therefore, timely detection and repair of damage can effectively reduce the risk of traffic accidents and injuries. By regularly inspecting and repairing damaged pavement, small problems can be prevented from expanding into big problems, thereby extending the service life of the road and reducing maintenance costs. Damaged crosswalks may cause a decrease in the smoothness of vehicle and pedestrian traffic and affect the road's capacity. Detecting and repairing these damages can improve traffic smoothness and reduce traffic congestion. Intact pavement is not only safer for pedestrians, but also improves the overall appearance of the city and the comfort of road use. Timely repair of damaged pavement helps maintain the image of the city.

[0047] At present, many damage detections rely on manual inspections, and staff judge the degree of road damage based on experience. This method is highly subjective, prone to missed detection or misjudgment, and the accuracy and consistency of the detection results are poor. Manual inspections are inefficient, especially in large-scale highway networks, where it is difficult to cover all sections efficiently and quickly. This will result in some damage not being discovered and repaired in time. Traditional detection methods rely on regular inspections and cannot achieve real-time and dynamic monitoring of damage. This may result in damaged sections not being repaired for a long time, affecting pedestrian safety. Most existing detection technologies rely on a single damage detection method, making it difficult to fully obtain road health data. Without systematic data analysis, it is difficult to provide decision support for long-term maintenance and management. Although manual inspection costs are relatively low, regular manual inspections and repairs are still an expensive investment for large-scale highway networks. Especially in remote areas, detection and repair are more difficult and costly. Although some detection technologies use high-tech means such as ground sensors or drones, these technologies still have limitations and cannot cover all types of damage or accurately identify certain subtle road cracks and damage.

[0048] In view of the limitations of the above existing detection technologies, this embodiment provides a method for detecting damaged pedestrian crossings on a highway pavement, including:

[0049] Obtain a high-resolution video image of the road surface to be inspected, and extract several frames of pedestrian crossing area images;

[0050] Inputting the pedestrian crossing area image into a preset damage detection model to obtain suspected damaged areas and corresponding position coordinates, wherein the damage detection model is used to perform feature extraction, damage judgment, and damaged area segmentation on the pedestrian crossing area image;

[0051] Determining the degree of damage of the suspected damaged area, and determining the actual damaged area according to the degree of damage;

[0052] The detection results of the same real damaged area corresponding to multiple frames of pedestrian crossing area images are integrated to obtain the final damaged area, and the final damaged area is matched with the high-precision map to obtain accurate geographic location information, thereby completing the pedestrian crossing damage detection on the road surface to be detected.

[0053] Specifically, this embodiment first extracts the crosswalk area from the high-resolution highway video, and performs image segmentation and enhancement processing; then uses a convolutional neural network to detect suspected damaged areas, and combines the support vector machine algorithm to classify the degree of damage. To improve detection accuracy, this embodiment uses a multi-scale fusion strategy to integrate the detection results of different frames, and finally matches the damage information with a high-precision map to obtain the precise geographic location and upload it to the road condition management platform. This embodiment realizes the automated detection and positioning of damaged crosswalks, provides an accurate decision-making basis for road maintenance, and effectively improves road safety and maintenance efficiency.

[0054] Furthermore, the method further includes preprocessing the crosswalk area image before inputting the crosswalk area image into the predicted damage detection model, wherein the preprocessing includes:

[0055] Binarizing the crosswalk area image, performing shape matching on the binarized crosswalk area image, determining the precise position of the crosswalk area, and capturing a first crosswalk image in the high-resolution video image based on the precise position;

[0056] Performing image segmentation processing on the first crosswalk image to obtain a second crosswalk image;

[0057] Perform image enhancement and denoising processing on the second crosswalk image to obtain a third crosswalk image.

[0058] Furthermore, the damage detection model includes: a feature extraction module, a damage detection module, and a region segmentation module, wherein the feature extraction module is used to extract features of the crosswalk area image based on a convolutional neural network to obtain image features; the damage detection module is used to classify the image features for damage based on a classifier to obtain damage detection results; the region segmentation module is used to locate and segment the damaged area when damage is detected, and output the contour and position coordinates of the damaged area.

[0059] Further, judging the damage degree of the suspected damaged area and determining the real damaged area according to the damage degree includes:

[0060] Dividing the suspected damaged area into image sub-blocks;

[0061] Extracting texture features and shape features of the image sub-blocks respectively, and constructing feature vectors;

[0062] Processing the feature vector by a support vector machine to obtain the damage degree of the suspected damaged area;

[0063] If the damage degree exceeds a preset damage threshold, the suspected damaged area is determined as a real damaged area; if the damage degree does not exceed the damage threshold, the suspected damaged area is determined as an undamaged area.

[0064] Furthermore, the detection results of the same real damaged area corresponding to multiple frames of pedestrian crossing area images are integrated, including:

[0065] Perform feature extraction on the same real damaged area detection results corresponding to multiple frames of pedestrian crossing area images to obtain multi-scale feature vectors;

[0066] The multi-scale feature vectors are fused, and damage identification and damage degree judgment are performed based on the fused feature vectors to obtain the final damaged area.

[0067] Furthermore, matching the final damaged area with a high-precision map to obtain accurate geographic location information includes:

[0068] Mapping the location coordinates of the final damaged area to the high-precision map and marking the damaged points;

[0069] The damaged points are grouped by a clustering algorithm to identify the densely-damaged areas and obtain the precise geographic location information.

[0070] Combine the following Figure 1-Figure 3 A method for detecting damage to a pedestrian crossing on a highway pavement proposed in this embodiment is described in detail, and specifically includes the following steps:

[0071] Step S101, obtaining a high-resolution video image of the road surface to be detected, and extracting a plurality of frames of pedestrian crossing area images, specifically comprising:

[0072] like Figure 2 As shown, a plurality of frames of pedestrian crossing area images are initially extracted from a high-resolution video image of a road surface to be detected, the pedestrian crossing area images are binarized, and shape matching is performed on the binarized pedestrian crossing area images to determine the precise position of the pedestrian crossing area, and a first pedestrian crossing image is captured in the high-resolution video image through the precise position; image segmentation is performed on the first pedestrian crossing image to obtain a second pedestrian crossing image; image enhancement and denoising are performed on the second pedestrian crossing image to obtain a third pedestrian crossing image.

[0073] Specifically, in this embodiment, after obtaining a high-resolution highway road surface video image, a crosswalk area image is extracted from the video image through image processing technology; the extracted crosswalk area image is binarized to obtain a binary crosswalk area image; based on the shape features of the crosswalk area, shape matching is performed on the binary crosswalk area image to determine the precise position of the crosswalk area; based on the determined precise position of the crosswalk area, the corresponding crosswalk area image is cut out from the original video image to obtain a first crosswalk image.

[0074] This embodiment uses a 4K high-definition camera installed on a road monitoring pole to capture video at a rate of 30 frames per second, uses a target detection algorithm based on YOLO (You Only Look Once) to identify the crosswalk area in the image, and uses image processing technology to cut out a rectangular area containing the crosswalk from the original video image. For example, the YOLO algorithm detects that the crosswalk is located in the upper left corner of the image, extracts the image of the area, and is a rectangular area of ​​200x300 pixels; binarization is performed on the extracted crosswalk area image to obtain a binarized crosswalk area image, and the binarization process uses the Otsu algorithm to automatically determine the optimal threshold, converting the image into black and white, with white pixels representing the white stripes of the crosswalk and black pixels representing the road surface; uses algorithms such as template matching or Hough transform to accurately locate the edges and corners of the crosswalk in the binarized image; based on the determined precise position of the crosswalk area, the corresponding crosswalk area image is cut out from the original video image to obtain a first crosswalk image.

[0075] Binarization is a common operation in image processing, which is used to convert grayscale images into black and white images. Through this process, each pixel in the image is divided into two categories according to a certain threshold: foreground (usually white) and background (usually black). The goal of binarization is to convert a grayscale image into an image with only two values, thereby simplifying subsequent processing steps such as image segmentation, feature extraction, etc.

[0076] The formula for binarization is as follows:

[0077]

[0078] Among them, I binary (x, y) represents the pixel value after binarization, usually 0 or 1, 0 represents black (background), 1 represents white (foreground), I(x, y) represents the grayscale value of the pixel at the coordinate (x, y) in the image, and T is the set threshold.

[0079] The first crosswalk image is classified at the pixel level through the pre-trained semantic segmentation model, and each pixel in the image is divided into a crosswalk area or a background area; according to the classification result, the first crosswalk image is segmented, and the pixels of the crosswalk area are extracted to obtain a segmented crosswalk area image; the segmented crosswalk area image is morphologically processed to eliminate noise and holes in the image to obtain a complete crosswalk area; the segmented crosswalk area is fused with the original image to obtain a second crosswalk image, in which the crosswalk area is presented in a highlighted manner. The semantic segmentation model is trained with a crosswalk image dataset, and the crosswalk image data contains crosswalk images under various lighting and weather conditions.

[0080] In this embodiment, the U-Net network structure is used to train the semantic segmentation model. The first pedestrian crossing image is input into the model. The model will classify the image at the pixel level to distinguish the pedestrian crossing and non-pedestrian crossing areas. Next, the classified image is morphologically processed, such as using dilation and corrosion operations, which can effectively remove small noise points and fill holes in the image, making the pedestrian crossing area more complete and clear. The processed pedestrian crossing area is merged with the original image, and a highlight display technology is usually used, such as presenting the pedestrian crossing area in a brighter color or increasing the brightness. This not only makes the visual effect more prominent, but also facilitates pedestrians and drivers to quickly identify.

[0081] The second pedestrian crossing image is converted into a grayscale image format and processed by a histogram equalization algorithm. By stretching the grayscale histogram of the image, the grayscale distribution of the image is made more uniform, thereby improving the contrast and clarity of the image. The image after histogram equalization is denoised using a median filter algorithm. Median filtering can effectively remove salt and pepper noise and speckle noise in the image by replacing the grayscale value of the pixel with the median of the grayscale value of the pixel's neighborhood. The filter window size of the median filter is determined according to the image size. Generally, an odd square matrix of 3x3 or 5x5 is selected as the filter window. The larger the filter window, the more obvious the denoising effect, but it may also cause the loss of image details. Edge detection is performed on the filtered image, and the gradient amplitude and direction of the image are calculated by the sobel operator to obtain the contour information of the image. The gradient amplitude is compared with the preset threshold. Pixels above the threshold are retained as edges, and pixels below the threshold are suppressed. According to the results of edge detection, the image is binarized. The grayscale values ​​of the pixels belonging to the edge are set to 255, and the grayscale values ​​of the remaining pixels are set to 0, so as to obtain a binary image containing only black and white colors. The binary image is morphologically processed, and the small connected areas and holes in the image are eliminated through corrosion and expansion operations, so that the crosswalk area is more complete and coherent, and finally the enhanced third crosswalk image is obtained.

[0082] In this embodiment, after the second crosswalk image is converted into a grayscale image, there is only one channel, and the pixel values ​​are still between 0 and 255, but it represents the brightness information of the image. Histogram equalization is a commonly used image enhancement technology, which can improve the contrast of the image and make the image details clearer and more visible. For example, if the grayscale histogram of the second crosswalk image is concentrated in a narrow range, for example, the grayscale levels of most pixel values ​​are concentrated between 100 and 150, then the image will appear gray and lack a sense of hierarchy. After histogram equalization processing, the distribution of gray levels will be more uniform.

[0083] The steps of histogram equalization include:

[0084] (1) Calculate the grayscale histogram of the image:

[0085]

[0086] Among them, p(i) is the probability of the pixel with gray value i appearing, N i is the number of pixels with gray value i, and N is the total number of pixels in the image.

[0087] (2) Calculate the cumulative distribution function:

[0088]

[0089] Where C(i) is the cumulative distribution function, p(k) is the probability of occurrence of a pixel with gray value k, k = 0, 1, 2, …, i, …, L-1, and L is the number of gray levels.

[0090] (3) Normalized cumulative distribution function:

[0091]

[0092] Among them, C min is the minimum value of the non-zero cumulative probability, and T(i) is the normalized cumulative distribution function.

[0093] (4) Mapping the original grayscale value to the new grayscale value:

[0094] I new (x,y)=T(I(x,y))

[0095] Among them, I(x,y) represents the gray value of the pixel at the coordinate (x,y) in the image, I new (x,y) represents the new grayscale value of the pixel at coordinate (x,y) in the image.

[0096] Median filtering is a nonlinear filtering technique that can effectively remove salt and pepper noise and speckle noise in an image while retaining the edge details of the image. The specific steps are: select a window size, which can be 3x3, 5x5, etc. The specific window size is selected according to the required denoising degree; for each pixel, extract all pixel values ​​in the window around the pixel; sort these pixel values ​​and take the median; replace the pixel value in the center of the window with the median.

[0097] The Sobel operator is a commonly used edge detection operator that can detect the edge in an image by calculating the gradient of the image. It uses two 3x3 convolution kernels to calculate the gradients in the horizontal and vertical directions respectively. After calculation by the Sobel operator, the horizontal and vertical gradients of the pixel can be obtained. According to the amplitude and direction of the gradient, it is judged whether the pixel belongs to the edge. The result of edge detection is used as the basis for binarization, and the image processing method based on set theory is used to change the shape and structure of the image. The small noise points and burrs are eliminated by the corrosion operation, and the small holes and cracks are filled by the expansion operation. The corrosion operation of the binary image can eliminate some isolated noise points on the edge of the crosswalk and make the edge smoother, while the expansion operation can connect some small cracks in the crosswalk area to make the crosswalk area more complete. The crosswalk area after such processing is clearer, which is conducive to subsequent recognition and analysis.

[0098] Step S102: input the pedestrian crossing area image into a preset damage detection model to obtain the suspected damaged area and the corresponding position coordinates.

[0099] The damage detection model includes: a feature extraction module, a damage detection module, and a region segmentation module, wherein the feature extraction module is used to extract features of the pedestrian crossing area image based on a convolutional neural network to obtain image features; the damage detection module is used to classify the image features for damage based on a classifier to obtain damage detection results; the region segmentation module is used to locate and segment the damaged area when damage is detected, and output the outline and position coordinates of the damaged area.

[0100] Specifically, in this embodiment, according to the pre-established damage detection model, a convolutional neural network algorithm, such as VGG or ResNet, is used to pre-process the third pedestrian crossing image and extract image features; the extracted image features are input into a pre-trained classifier for damaged area detection to determine whether there are suspected damaged areas. For example, if the texture features of an area are similar to the crack features in the training set, the classifier may mark it as a suspected damaged area. Once the suspected damaged areas are detected, the edge detection algorithm is used to accurately locate the contours of these areas. The edge detection algorithm depicts the contours of objects in the image by identifying sudden changes in pixel intensity, which is very useful for determining the exact boundaries of the damaged areas. After determining the location of the damaged areas, image segmentation techniques, such as threshold segmentation or region growing algorithms, are used to further isolate these areas. These techniques divide the image by analyzing the similarity between pixels, thereby forming clear contours of the damaged areas.

[0101] The edge detection algorithm can use the Canny edge detection algorithm. The Canny algorithm first performs Gaussian blur on the image to reduce the impact of noise, then calculates the gradient amplitude and direction of the image, then performs non-maximum suppression to refine the edge, and finally uses the double threshold method to detect and connect the edges. For example, assuming that there is a damaged area in the crosswalk image that is significantly darker than the surrounding area, the Canny algorithm can effectively detect the edge of the damaged area and form a closed contour. After obtaining the contour of the damaged area through the edge detection algorithm, it is necessary to use the contour extraction algorithm to obtain the precise contour coordinates.

[0102] The contour extraction algorithm uses the Suzuki85 algorithm to connect the edge points to form an ordered coordinate sequence and store it in an array. For example, assuming that the edge contour of the damaged area is a rectangle, the contour extraction algorithm can obtain the coordinates of the four vertices of the rectangle and store them in an array, such as [(10,20), (30,20), (30,50), (10,50)]. Based on the contour coordinate array, the region growing algorithm can be used to accurately segment the damaged area.

[0103] The region growing algorithm starts from a seed point and merges the adjacent pixels with similar features into the same region until there are no new pixels to be merged. In this embodiment, any point inside the damaged area is used as a seed point, and all the pixels within the outline are merged into the same region, thereby achieving the segmentation of the damaged area.

[0104] Step S103: determine the damage degree of the suspected damaged area, and determine the real damaged area according to the damage degree.

[0105] like Figure 3As shown, the suspected damaged area is divided into image sub-blocks; the texture features and shape features of the image sub-blocks are extracted respectively, and a feature vector is constructed; the feature vector is processed by a support vector machine to obtain the damage degree of the suspected damaged area; if the damage degree exceeds a preset damage threshold, the suspected damaged area is determined to be a real damaged area; if the damage degree does not exceed the damage threshold, the suspected damaged area is determined to be an undamaged area.

[0106] Specifically, in this embodiment, based on the pixel coordinates of the suspected damaged area, such as the rectangular area from (100,50) to (150,100), the image sub-block of the area is extracted from the original crosswalk image. Then, the grayscale co-occurrence matrix algorithm is used to extract the texture features of the image sub-block. For example, by calculating the grayscale co-occurrence matrix of adjacent pixels in the horizontal direction, features such as a contrast of 5, a correlation of 0.8, and an entropy of 2 can be obtained. These features reflect the law of grayscale changes of pixels within the image sub-block. The contrast of the crack area is usually higher, while the entropy value of the pothole area may be larger. At the same time, the shape features of the image sub-block are obtained by the contour extraction algorithm. For example, the area of ​​the area is calculated to be 2000 pixels, the perimeter is 200 pixels, and the circularity is 0.6. Shape features can distinguish different types of damage. For example, the perimeter of the crack area is usually longer, while the area of ​​the pothole area may be larger. The extracted texture features (contrast, correlation, entropy) and shape features (area, perimeter, circularity) are constructed into a feature vector, such as [5, 0.8, 2, 2000, 200, 0.6]. This feature vector comprehensively reflects the texture and shape information of the suspected damaged area. The feature vector is input into the pre-trained support vector machine classification model. This model has been trained using a pre-established damage feature model and can classify the input feature vector into different damage levels, such as slight damage, moderate damage, and severe damage. Assume that the classification result of the suspected damaged area is moderate damage. A damage severity threshold is pre-set, such as moderate damage. If the classification result exceeds the preset threshold, such as severe damage, the suspected damaged area is determined to be a real damaged area, and its position coordinates and damage degree information are output. If the classification result does not exceed the threshold, such as slight damage, the suspected damaged area is determined to be a normal area and no subsequent processing is performed.

[0107] The texture features of image sub-blocks extracted using the gray level co-occurrence matrix algorithm include:

[0108] (1) Constructing gray-level co-occurrence matrix: Calculate the gray-level co-occurrence matrix for the image sub-block. The gray-level co-occurrence matrix is ​​a two-dimensional matrix. Each element M(i, j) represents the number of occurrences of gray-level values ​​i and j at a specified direction and distance. The size of the matrix is ​​L×L, where L is the number of gray levels.

[0109] (2) Normalized gray-level co-occurrence matrix:

[0110]

[0111] Among them, P(i,j) represents the normalized gray-level co-occurrence matrix.

[0112] (3) Extracting texture features: The contrast, correlation, energy, entropy, uniformity, etc. are calculated through the normalized gray-level co-occurrence matrix to obtain the extracted texture features.

[0113] The shape features of image sub-blocks obtained by contour extraction algorithm include:

[0114] (1) Use edge detection algorithm to extract contours, such as Canny edge detection algorithm and Sobel operator. After obtaining the contour of the damaged area through the edge detection algorithm, it is necessary to use the contour extraction algorithm to obtain the precise contour coordinates. The contour extraction algorithm uses the Suzuki85 algorithm to connect the edge points to form an ordered coordinate sequence and store it in an array.

[0115] (2) Calculation of shape features: The perimeter, area, circularity, moment, aspect ratio, convex hull, etc. are calculated through the extracted contour, which are the extracted shape features.

[0116] Step S104: Integrate the detection results of the same real damaged area corresponding to multiple frames of pedestrian crossing area images to obtain a final damaged area.

[0117] Feature extraction is performed on the detection results of the same real damaged area corresponding to the multiple frames of pedestrian crossing area images to obtain a multi-scale feature vector; the multi-scale feature vectors are fused, and damage identification and damage degree judgment are performed based on the fused feature vectors to obtain the final damaged area.

[0118] Specifically, in this embodiment, a multi-scale image pyramid technique is used to observe the changes in damage features by gradually reducing the resolution of the image, thereby extracting feature vectors at different scales. The feature vectors extracted at different scales are then fused to output a comprehensive feature vector. The fused feature vector can more comprehensively reflect the characteristics of the damaged area. The fusion method can be to splice feature vectors of different scales together, or take an average value, etc. The purpose of fusing feature vectors is to integrate information at different scales and improve the accuracy of damage detection. For example, the feature vectors [0.2, 0.5, 0.8] and [0.3, 0.6, 0.9] at two scales of 500x500 and 250x250 are spliced ​​into [0.2, 0.5, 0.8, 0.3, 0.6, 0.9], or the average value is taken to obtain [0.25, 0.55, 0.85]. The fused feature vector is processed again by a support vector machine to obtain the degree of damage and the position and size of the final damaged area.

[0119] The multi-scale image pyramid technology is used to observe the changes in damage features by gradually reducing the resolution of the image. The feature vectors are extracted at different scales, including: (1) constructing a pyramid hierarchy based on the image, with the resolution of each layer of the image being lower than that of the previous layer; (2) extracting features at each layer of the pyramid hierarchy, which can adopt methods such as scale-invariant feature transformation, accelerated robust features, oriented gradient histograms, and convolutional neural networks.

[0120] Step S105: Match the final damaged area with the high-precision map to obtain accurate geographic location information, and complete the pedestrian crossing damage detection on the road surface to be detected.

[0121] The location coordinates of the final damaged area are mapped to the high-precision map, and the damaged points are marked; the damaged points are grouped by a clustering algorithm, the densely damaged areas are identified, and the precise geographic location information is obtained.

[0122] Specifically, in this embodiment, the pixel coordinates of the final damaged area are converted into geographic coordinates in the map by using the internal and external parameters of the camera and high-precision map data. The internal and external parameters of the camera describe the geometric relationship between the camera and the real world, and the high-precision map provides a mapping relationship between pixel coordinates and geographic coordinates. Assuming that the height of the camera is 2 meters and the shooting angle is 45 degrees, through projection transformation and map matching, the pixel coordinates (100,50) and (150,100) can be converted into a circular area with a radius of 1 meter as the center and the geographic coordinates (31.22°N, 121.48°E). The geographic coordinates and attribute information of the final damaged area are associated to generate a structured damage data record, which may contain the damaged geographic coordinates (31.22°N, 121.48°E), type (crack), severity (medium) and other information.

[0123] Match the damage data records with high-precision map data, and mark the location of the damage points on the map; use clustering algorithms, such as the DBSCAN algorithm, to group the damage points and identify the densely damaged areas on the road. For example, damage points with similar geographical locations and similar damage types are grouped together. Assuming that on a certain road section, multiple crack-type damage points are found to be clustered together, the section can be marked as a densely damaged area. This helps to determine the key sections for road maintenance and concentrate resources for repairs. Comprehensively analyze factors such as damage type, severity, and density, and generate road maintenance priority ranking and maintenance plan recommendations through a decision tree model. For example, a higher priority is given to sections with severe damage and high density. The decision tree model can automatically generate maintenance plan recommendations based on pre-set rules, such as "If the damage type is potholes and the severity is severe, the priority is the highest."

[0124] To further optimize the above technical solution, this embodiment provides a road pavement pedestrian crossing damage detection system, such as Figure 4 As shown, it includes: a test image acquisition module, a damaged area detection module, and a damaged area integration module;

[0125] The image acquisition module to be tested is used to acquire a high-resolution video image of the road surface to be tested and extract a plurality of frames of images of the crosswalk area;

[0126] The damaged area detection module is used to input the pedestrian crossing area image into a preset damage detection model, obtain the suspected damaged area and the corresponding position coordinates, and determine the damage degree of the suspected damaged area, and determine the real damaged area according to the damage degree;

[0127] The damaged area integration module is used to integrate the detection results of the same real damaged area corresponding to multiple frames of pedestrian crossing area images to obtain the final damaged area, and match the final damaged area with the high-precision map to obtain accurate geographic location information to complete the pedestrian crossing damage detection on the highway road surface to be detected.

[0128] Furthermore, if Figure 5 As shown, the image acquisition module to be tested includes: a video segmentation unit and an image processing unit, wherein the video segmentation unit is used to segment and extract a plurality of frames of pedestrian crossing area images of different scales from the high-resolution video image of the road surface to be tested; and the image processing unit is used to perform binarization, image segmentation, image enhancement, and denoising on the pedestrian crossing area image to obtain the processed pedestrian crossing area image.

[0129] Furthermore, if Figure 6As shown, the damaged area detection module includes: a damaged area detection unit and a damage degree judgment unit, wherein the damaged area detection unit is used to perform feature extraction, feature recognition, and area segmentation on the pedestrian crossing area image based on the damage detection model, and output the contour and position coordinates of the suspected damaged area; the damage degree judgment unit is used to extract the texture features and shape features of the suspected damaged area, judge the damage degree based on the texture features and shape features, and determine the real damaged area according to the damage degree.

[0130] Furthermore, if Figure 7 As shown, the damaged area integration module includes: a multi-scale fusion unit and a high-precision positioning unit, wherein the multi-scale fusion unit is used to extract features from the same real damaged area detection results corresponding to multiple frames of pedestrian crossing area images, obtain multi-scale feature vectors and fuse them, and identify the final damaged area based on the fused feature vectors; the high-precision positioning unit is used to map the position coordinates of the final damaged area to a high-precision map and mark the damaged points, group the damaged points through a clustering algorithm, identify the densely damaged area and obtain precise geographic location information.

[0131] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for detecting damage to a pedestrian crossing on a highway pavement, characterized in that: include: Obtain a high-resolution video image of the road surface to be inspected, and extract several frames of pedestrian crossing area images; Inputting the pedestrian crossing area image into a preset damage detection model to obtain suspected damaged areas and corresponding position coordinates, wherein the damage detection model is used to perform feature extraction, damage judgment, and damaged area segmentation on the pedestrian crossing area image; Determining the damage degree of the suspected damaged area, and determining the real damaged area according to the damage degree; The detection results of the same real damaged area corresponding to multiple frames of pedestrian crossing area images are integrated to obtain the final damaged area, and the final damaged area is matched with the high-precision map to obtain accurate geographic location information, thereby completing the pedestrian crossing damage detection on the road surface to be detected.

2. The method for detecting damage to a pedestrian crossing on a highway pavement according to claim 1, characterized in that: Before inputting the pedestrian crossing area image into the predicted damage detection model, the pedestrian crossing area image is also preprocessed, and the preprocessing includes: Binarizing the crosswalk area image, performing shape matching on the binarized crosswalk area image, determining the precise position of the crosswalk area, and capturing a first crosswalk image in the high-resolution video image based on the precise position; Performing image segmentation processing on the first crosswalk image to obtain a second crosswalk image; Perform image enhancement and denoising processing on the second crosswalk image to obtain a third crosswalk image.

3. The method for detecting damage to a pedestrian crossing on a highway pavement according to claim 1, characterized in that: The damage detection model includes: a feature extraction module, a damage detection module, and a region segmentation module, wherein the feature extraction module is used to extract features of the pedestrian crossing area image based on a convolutional neural network to obtain image features; the damage detection module is used to classify the image features for damage based on a classifier to obtain damage detection results; the region segmentation module is used to locate and segment the damaged area when damage is detected, and output the outline and position coordinates of the damaged area.

4. The method for detecting damage to a pedestrian crossing on a highway pavement according to claim 1, characterized in that: Determining the damage degree of the suspected damaged area and determining the real damaged area according to the damage degree includes: Dividing the suspected damaged area into image sub-blocks; Extracting texture features and shape features of the image sub-blocks respectively, and constructing feature vectors; Processing the feature vector by a support vector machine to obtain the damage degree of the suspected damaged area; If the damage degree exceeds a preset damage threshold, the suspected damaged area is determined as a real damaged area; if the damage degree does not exceed the damage threshold, the suspected damaged area is determined as an undamaged area.

5. The method for detecting damage to a pedestrian crossing on a highway pavement according to claim 1, characterized in that: Integrating the detection results of the same real damaged area corresponding to multiple frames of pedestrian crossing area images includes: Perform feature extraction on the same real damaged area detection results corresponding to multiple frames of pedestrian crossing area images to obtain multi-scale feature vectors; The multi-scale feature vectors are fused, and damage identification and damage degree judgment are performed based on the fused feature vectors to obtain the final damaged area.

6. The method for detecting damage to a pedestrian crossing on a highway pavement according to claim 1, characterized in that: Matching the final damaged area with the high-precision map to obtain accurate geographic location information includes: Mapping the location coordinates of the final damaged area to the high-precision map and marking the damaged points; The damaged points are grouped by a clustering algorithm to identify the densely-damaged areas and obtain the precise geographic location information.

7. A road pavement pedestrian crossing damage detection system, characterized in that: include: Test image acquisition module, damaged area detection module, damaged area integration module; The image acquisition module to be tested is used to acquire a high-resolution video image of the road surface to be tested and extract a plurality of frames of images of the crosswalk area; The damaged area detection module is used to input the pedestrian crossing area image into a preset damage detection model, obtain the suspected damaged area and the corresponding position coordinates, and determine the damage degree of the suspected damaged area, and determine the real damaged area according to the damage degree; The damaged area integration module is used to integrate the detection results of the same real damaged area corresponding to multiple frames of pedestrian crossing area images to obtain the final damaged area, and match the final damaged area with the high-precision map to obtain accurate geographic location information to complete the pedestrian crossing damage detection on the highway road surface to be detected.

8. The highway pavement pedestrian crossing damage detection system according to claim 7, characterized in that: The image acquisition module to be tested includes: a video segmentation unit and an image processing unit, wherein the video segmentation unit is used to segment and extract a plurality of frames of pedestrian crossing area images of different scales from the high-resolution video image of the road surface to be tested; and the image processing unit is used to perform binarization, image segmentation, image enhancement, and denoising on the pedestrian crossing area image to obtain the processed pedestrian crossing area image.

9. The highway pavement pedestrian crossing damage detection system according to claim 7, characterized in that: The damaged area detection module includes: a damaged area detection unit and a damage degree judgment unit, wherein the damaged area detection unit is used to perform feature extraction, feature recognition, and area segmentation on the pedestrian crossing area image based on the damage detection model, and output the outline and position coordinates of the suspected damaged area; the damage degree judgment unit is used to extract the texture features and shape features of the suspected damaged area, judge the degree of damage based on the texture features and shape features, and determine the real damaged area according to the degree of damage.

10. The highway pavement pedestrian crossing damage detection system according to claim 7, characterized in that: The damaged area integration module includes: a multi-scale fusion unit and a high-precision positioning unit, wherein the multi-scale fusion unit is used to extract features from the same real damaged area detection results corresponding to multiple frames of pedestrian crossing area images, obtain multi-scale feature vectors and fuse them, and identify the final damaged area based on the fused feature vectors; the high-precision positioning unit is used to map the location coordinates of the final damaged area to a high-precision map and mark the damaged points, group the damaged points through a clustering algorithm, identify the densely damaged areas and obtain precise geographic location information.

Images