A highway guardrail extraction method
By improving the distance model in the HSV space and combining morphological reconstruction, contour area method, Gabor filter and superpixel segmentation, the robustness and accuracy of highway guardrail extraction are solved, and efficient and complete guardrail image segmentation is achieved.
Patent Information
- Application Number
- CN202211331232.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-10-28
- Publication Date
- 2025-09-02
- Estimated Expiration
- 2042-10-28
AI Technical Summary
The prior art is not robust and accurate in road guardrail extraction and is susceptible to environmental impact, especially through deep learning methods, which require a large number of data sets to be trained and have poor network structure adaptability.
Binarized images were initially extracted in HSV space through improved distance model, combined with morphological reconstruction and contour area method to remove noise, and used fusion Gabor filter and superpixel segmentation to remove similar semantics to realize the extraction of road guardrails.
The robustness and accuracy of highway guardrail extraction are improved, and the segmented guardrail image structure is complete, laying a good foundation for subsequent inspection.
Smart Images
Figure CN115578629B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of digital image processing, and more particularly to a method for extracting highway guardrails. Background Art
[0002] Digital image processing refers to the methods and techniques of removing noise, enhancing, restoring, segmenting, extracting features, etc. from images through computers.
[0003] In recent years, with the continuous development of machine vision and artificial intelligence in the transportation sector, various nondestructive testing systems and research methods have emerged. Highway guardrails, as part of road infrastructure, are primarily installed on both sides of the road. They set clear zones to reduce the risk of vehicles exiting or entering oncoming lanes. Therefore, they play a vital role in protecting drivers and reducing traffic accidents. Efficient detection and identification of these guardrails can more effectively safeguard drivers' safety.
[0004] Currently, the extraction of road guardrails can be roughly divided into two categories. One is to extract the structure of road guardrails through lidar and scanning lines, but this method often cannot fully obtain the integrity of the structure and is easily affected by the environment; the second is through deep learning and image processing methods, mainly based on deep learning, but deep learning requires a large amount of data sets for training, and the adaptability of the network structure needs to be constantly adjusted, and its robustness and accuracy are not very high.
[0005] Therefore, how to improve the accuracy and robustness of target extraction is an urgent problem that those skilled in the art need to solve. Summary of the Invention
[0006] In view of this, the present invention provides a highway guardrail extraction method to solve the technical problems in the background technology.
[0007] In order to achieve the above object, the present invention provides the following technical solutions:
[0008] A highway guardrail extraction method, characterized in that the specific steps include:
[0009] Get the image to be processed;
[0010] Perform color quantization on the region of interest to obtain the HSV histogram of the ROI region, and use the average values of the H, S, and V components of the region of interest as the H, S, and V components of the target pixel;
[0011] Calculate the distance between the H, S, and V components of each pixel and the corresponding components of the target pixel, and determine the initial distance threshold. Iterate and select the color range of the target area according to the initial distance threshold for preliminary extraction to obtain a binary image.
[0012] The obtained binary image is subjected to morphological reconstruction and contour area method to remove interference noise;
[0013] Superpixel segmentation is performed based on texture features combined with Gabor filters to remove similar semantics;
[0014] Get the image of interest.
[0015] Optionally, in the above-mentioned highway guardrail extraction method, the specific steps of converting the image to be processed from the RGB color space to the HSV space are as follows:
[0016]
[0017]
[0018]
[0019] Optionally, in the above-mentioned highway guardrail extraction method, the specific steps of initially extracting and obtaining a binary image are as follows:
[0020] Perform color quantification on the region of interest to obtain the HSV histogram of the ROI area;
[0021] The components of each pixel are recorded as H, S, V, and the distance formula of the target pixel is calculated using the improved distance formula, and all the distances obtained are normalized;
[0022] The average value of all distances is used as the initialization distance threshold, and distances smaller than the initialization distance threshold are continuously screened. A new distance threshold is calculated each time an iteration is performed, and the next iteration is continued on this basis until all distances are smaller than the initialization distance threshold.
[0023] The distance that finally meets the conditions is matched one-to-one with the H, S, and V components of the corresponding pixel points, and the maximum and minimum values of each pixel component are obtained as the color range of the target area. The pixels greater than the maximum component and less than the minimum component are set to 0, and the pixels greater than the minimum threshold and less than the maximum threshold are set to 255 to obtain the binary image of the target area.
[0024] Optionally, in the above-mentioned highway guardrail extraction method, the distance formula is calculated for the components of the target pixel point using an improved distance formula:
[0025]
[0026] Among them, H, S, V represent the H, S, V components in the HSV space respectively; H0, S0, V0 represent the target components H0, S0, V0 of the target pixel point.
[0027] Furthermore, based on the above-mentioned elimination of a large number of interference semantics through morphological reconstruction and contour area method, traditional morphological operations will cause image blur, consciously change the morphological characteristics of the target structure, cause deformation of the segmented area, increase or decrease or disconnect the connected domain, and the reconstruction morphological filtering can remove local grayscale peaks without changing the region boundary, and retain components larger than the structural elements, thereby preserving the integrity of the structure. The morphological reconstruction method can effectively fill the holes in the target area and reduce the influence of noise in the target area.
[0028] Optionally, in the above-mentioned highway guardrail extraction method, the specific steps of morphological reconstruction are as follows:
[0029] The complement of the obtained binary image is recorded as I c :
[0030]
[0031] And form a marked image with a boundary of 1, denoted as F:
[0032]
[0033] Assuming that the structural element to be reconstructed is B, the following operations are performed to remove isolated noise points in the image:
[0034]
[0035] Optionally, in the above-mentioned highway guardrail extraction method, the specific steps of the contour area method are as follows:
[0036] The morphologically reconstructed image is used to remove small areas using the contour area method, and the area of other pixels is compared with the area of the largest pixel, and this ratio is set as K;
[0037]
[0038] The contour area that is less than or equal to the preset threshold is eliminated, and the processed image is then multiplied with the image to be processed to obtain the image of the target area.
[0039] Furthermore, disconnected areas and interference noises other than the target area can be removed without destroying the overall structure.
[0040] It is important to understand that the Gabor filter is composed of a set of wavelets that capture energy of a specific frequency and direction through wavelets of different scales and directions. Gabor only allows textures of the corresponding frequency to pass through, filters out textures of other frequencies, and extracts the corresponding texture features. However, texture images with only one direction or one frequency are very limited.
[0041] Therefore, the present invention combines images of different directions and frequencies to solve the limitations caused by single direction and single frequency. Filtering is performed in four different directions and six different Gabor kernels, generating a total of 24 sets of different texture filter images. The images of different directions and frequencies are weighted fused, and the average value of the weighted texture image is calculated as the initial seed point of the texture vector. The image is then converted to Lab space, and the three color feature vectors in the Lab space, the coordinates of the pixel point in space, and the initial seed point of the texture image are combined to form a six-dimensional feature vector a=[l k , a k , b k , x k ,y k , t k ] T , perform similarity clustering on the pixels according to the six-dimensional feature vector, and calculate the distance D between the pixels around the cluster center and the pixel center k (p):
[0042]
[0043] It then iterates and calculates duplicate pixels and residual error until the residual error falls below a threshold. Different textures can vary significantly in appearance and brightness. Incorporating texture features into the SLIC algorithm for image segmentation makes the image's brightness and spatial characteristics more sensitive. Using a superpixel search approach, thresholding and iterating pixels around the center point effectively removes similar semantics and produces a ROI image.
[0044] As can be seen from the above technical solutions, compared with the existing technology, the present invention provides a highway guardrail extraction method. A binary image is initially extracted using an improved distance model in HSV space. Noise is removed primarily through morphological reconstruction and the contour area method. A fusion of Gabor and superpixels is used to remove similar semantics, thereby achieving highway guardrail extraction. Due to the complex background of highway guardrails, the present invention has significant advantages over existing technologies in terms of high robustness and detection and recognition accuracy. The segmented guardrail image is also structurally complete, laying a good foundation for subsequent detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0046] Figure 1 This is the overall flow chart of the highway guardrail extraction proposed by the present invention;
[0047] Figure 2 This is a flow chart of the adaptive binarization of an image in HSV space proposed by the present invention;
[0048] Figure 3 are the H, S, V histograms of the target image proposed by the present invention;
[0049] Figure 4 is a distribution diagram of the distance d after iteration proposed by the present invention;
[0050] Figure 5 is the binary image initially extracted by the present invention;
[0051] Figure 6 is the image after morphological reconstruction proposed by the present invention;
[0052] Figure 7 This is the image after the small area is extracted by the contour ratio method proposed in the present invention;
[0053] Figure 8 This is the superpixel segmentation flow chart combined with Gabor proposed in the present invention;
[0054] Figure 9 This is the comparison image after the present invention fuses Gabor and superpixel segmentation;
[0055] Figure 10 This is a comparison result between the present invention and the traditional method. DETAILED DESCRIPTION
[0056] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0057] The present invention discloses a method for extracting highway guardrails. This method uses an improved distance model in HSV space to initially extract a binary image. Noise is removed primarily through morphological reconstruction and the contour area method. Gabor and superpixel fusion is then used to remove similar semantics, thereby enabling the extraction of highway guardrails. Due to the complex background of highway guardrails, the present invention offers significant advantages over existing technologies, including greater robustness and high detection and recognition accuracy. The segmented guardrail image also possesses a relatively complete structural integrity, laying a solid foundation for subsequent detection.
[0058] The present invention discloses a method for extracting highway guardrails. Figure 1 As shown: preliminary extraction of binary image; removal of independent noise points; removal of connected domain noise points.
[0059] The following is a detailed description of the specific design steps and processing procedures of the highway guardrail extraction method through a specific example.
[0060] 1. Preliminary extraction of binary image
[0061] (1) The flowchart of adaptive extraction of binary image is as follows Figure 2 As shown, we first need to input an image. Since highway guardrails have obvious color features, we can segment the target image by extracting the target's color. Therefore, we first convert the image from RGB color space to HSV space for processing. The conversion formula is as follows:
[0062]
[0063]
[0064]
[0065] (2) Perform color quantization on the region of interest and obtain the HSV histogram of the ROI region as follows Figure 3 Since the hue, brightness and saturation of the ROI area are mostly concentrated in one range, that is, the range of the target color, and since the color information is mainly concentrated in the low-order moments, the first-order moment in the color moment is used to represent the main color information of the ROI area:
[0066]
[0067]
[0068]
[0069] (3) The components of the surrounding pixels are recorded as H, S, and V, and the distance formula for the components of the target pixel is calculated using the improved distance formula as follows:
[0070]
[0071] All distances obtained are normalized, that is, all distances are normalized to between (0, 1).
[0072] (4) The average of all distances is used as the initialization distance threshold d m , continuously screening less than d m A new distance threshold is calculated each time it is iterated, and the next iteration is continued on this basis until all distances d are less than the initialization distance threshold d m , the distance d after iteration is as follows Figure 4 shown.
[0073] (5) The distance that finally meets the conditions is matched with the H, S, and V components of the corresponding pixel points one by one, and the maximum and minimum values of each pixel component are obtained as the color range of the target area. The pixels greater than the maximum component and less than the minimum component are set to 0, and the pixels greater than the minimum threshold and less than the maximum threshold are set to 255, thereby preliminarily obtaining the binary image of the target area as shown in the figure. Figure 5 shown.
[0074] 2. Remove independent noise
[0075] (1) Figure 5 The extracted image shown in the figure has many small holes. In order to make the target area clearer without destroying the overall structure of the guardrail, the morphological reconstruction method is used to remove the holes and black spots on the target image. Morphological reconstruction mainly involves the interaction between two images and a structural element. Morphological reconstruction is used to remove holes in the ROI area. First, the complement of the binary image obtained above is calculated and recorded as I c :
[0076]
[0077] And form a marked image with a boundary of 1, denoted as F:
[0078]
[0079] Assuming that the structural element to be reconstructed is B, the following operations are performed to achieve the effect of removing holes in the image:
[0080]
[0081] Through morphological reconstruction, small holes are filled in the target area of the image to reduce the noise effect in the target area. The results are as follows: Figure 6 shown.
[0082] (2) After morphological reconstruction, there are still many noise points in the image. In order not to destroy the integrity of the target structure, the contour area method is used to eliminate small areas. The area of other pixels is compared with the area of the largest pixel, and this ratio is set as K. The contour areas with K≤0.2 are eliminated, and the processed image is multiplied with the image to be processed to obtain a composite image of the target area. The composite image is as follows: Figure 7 shown.
[0083] 3. Remove noise in connected areas
[0084] (1) Removing noise points in connected domains is mainly done by segmenting similar semantics through texture features. The main process is as follows: Figure 8 As shown in Figure 1, the Gabor filter captures energy at specific frequencies and directions through a set of wavelets of different scales and orientations. Gabor only allows textures at the corresponding frequencies to pass through, filtering out textures at other frequencies and extracting the corresponding texture features. Let the pixel in the image be I(x, y), and the two-dimensional Gabor transform is as follows:
[0085]
[0086] x′=xcosθ+ysinθ, y′=-xsinθ+ycosθ
[0087] Where f and θ represent the center frequency and direction respectively, σ x and σ y Represents the broadening of the Gaussian function in the x- and y-directions. Because different textures have different center frequencies and bandwidths, the Gabor filter only allows textures with the corresponding frequency to pass through, while textures with other frequencies are not allowed to pass through, thereby achieving the effect of suppressing other textures and extracting the corresponding texture features.
[0088] Texture images with one direction and one frequency are very limited. Therefore, this invention combines images with different directions and frequencies to overcome the limitations of single direction and frequency. Filtering was performed with four different directions (0°, 45°, 90°, and 135°) and six different Gabor kernels (7, 9, 11, 13, 15, and 17), generating a total of 24 sets of texture filter images. The images generated at different frequencies and directions are denoted as I f，θ , and fuse images of different directions and frequencies with Gaussian weighting:
[0089]
[0090] According to the results of the images, the synthesized images have better effects than single-channel, single-direction images.
[0091] (2) Next, the texture features are integrated into the SLIC algorithm. First, the average value of the texture image is calculated as the initial seed point of the texture vector. Then the image is converted to Lab space, and the three color feature vectors in the Lab space, the coordinates of the pixel point in space, and the initial seed point of the texture are combined to form a six-dimensional feature vector a = [l k , a k , b k , x k ,y k , t k ] T , similarity clustering of pixels is performed based on the six-dimensional feature vector.
[0092] There are N pixels in the synthetic image. For each pixel position, the initialization label L(p) = -1 and the initialization distance d(p) = ∞ are given. The K blocks divided into required areas will initialize the seed point of each block as the center of the cluster:
[0093] M k =[l k , a k , b k , x k ,y k , t k ]
[0094] Then the distance between each cluster center is Then calculate the cluster center M k The distance from the surrounding 2S×2S pixels to the cluster center:
[0095]
[0096] If D k <d(p) then update d(p) = D k , L(p)=k, and use C k Denote the set of pixels with label L(p) = k, forming a new cluster center:
[0097]
[0098] where |C k | is the set C k Recalculate the number of pixels and iterate to calculate the new C k and residual error, and the iteration stops when the residual error is less than the threshold.
[0099] Different textures can also have great differences in display and brightness. Integrating texture features into the SLIC algorithm for image segmentation makes the brightness and spatial features of the image more sensitive. The threshold value of the pixels around the center point is judged and iterated according to the superpixel search method. The comparison results of the segmentation are as follows: Figure 9 .
[0100] Since there are relatively few methods for highway guardrail extraction, the traditional Ostu method and Sobel method are selected to compare with the method of the present invention. The comparison results are as follows: Figure 10 shown.
[0101] In addition, the present invention compares the extraction accuracy of the three methods using three indicators: IOU, Dice, and Precision. The comparison results are shown in Table 1:
[0102] Table 1 Comparison of experimental results
[0103]
[0104] From the comparison of the data and images in the above table, it can be seen that the highway guardrail extraction method proposed in this invention uses an improved distance model in the HSV space and combines it with Gabor, superpixel segmentation and other methods to extract the final ROI area. This lays a certain foundation for more effective subsequent recognition and detection, thereby realizing the automatic extraction and non-destructive detection of highway guardrails.
[0105] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.
[0106] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A highway guardrail extraction method, characterized in that: The specific steps include: Get the image to be processed; Convert the image to be processed from RGB color space to HSV color space; Calculate the distance between the H, S, and V components of each pixel and the corresponding components of the target pixel, and determine the initial distance threshold. Iterate and select the color range of the target area according to the initial distance threshold for preliminary extraction to obtain a binary image. The obtained binary image is subjected to morphological reconstruction and contour area method to remove interference noise; Superpixel segmentation is performed based on texture features combined with Gabor filters to remove similar semantics; Get the image of interest; The specific steps for preliminary extraction to obtain a binary image are as follows: Perform color quantization on the region of interest to obtain the HSV histogram of the ROI region, and use the average values of the H, S, and V components of the region of interest as the H, S, and V components of the target pixel; The components of each pixel are recorded as H, S, V, and the distance formula of the target pixel is calculated using the improved distance formula, and all the distances obtained are normalized; The average value of all distances is used as the initialization distance threshold, and distances smaller than the initialization distance threshold are continuously screened. A new distance threshold is calculated each time an iteration is performed, and the next iteration is continued on this basis until all distances are smaller than the initialization distance threshold. The distance that finally meets the conditions is matched one-to-one with the H, S, and V components of the corresponding pixel points, and the maximum and minimum values of each pixel component are obtained as the color range of the target area. The pixels greater than the maximum component and less than the minimum component are set to 0, and the pixels greater than the minimum threshold and less than the maximum threshold are set to 255, thus obtaining a binary image of the target area. The specific steps of the contour area method are as follows: The morphologically reconstructed image is used to remove small areas using the contour area method, and the area of other pixels is compared with the area of the largest pixel, and this ratio is set as K; Eliminate contour areas that are smaller than or equal to a preset threshold, and perform dot multiplication of the image after contour area elimination with the image to be processed to obtain a composite image of the target area; The superpixel segmentation specifically involves the following steps: Eliminate the pixel points in the image whose contour area is less than or equal to the preset threshold and set them as I(x, y), perform two-dimensional Gabor transform, and extract the corresponding texture features through Gabor filter; x′=xcosθ+ysinθ, y′=-xsinθ+ycosθ; Where f and θ represent the center frequency and direction respectively, σ x and σ y Represents the broadening of the Gaussian function in the x-direction and the Y-direction; The generated texture images at different frequencies and directions are recorded as I f，θ , and Gaussian weighted fusion of texture images of different directions and frequencies: The texture features are integrated into the SLIC algorithm, and the average value of the texture image is calculated as the initial seed point of the texture vector; The texture image is converted into Lab space, and the three color feature vectors in the Lab space, the coordinates of the pixel point in the space and the initial seed point of the texture image form a six-dimensional feature vector a = [l k , a k , b k , x k ,y k , t k ] T , perform similarity clustering on pixels based on the six-dimensional feature vector; Output the segmented region of interest.
2. A highway guardrail extraction method according to claim 1, characterized in that: The specific steps of converting the image to be processed from RGB color space to HSV space are as follows:
3. A highway guardrail extraction method according to claim 2, characterized in that: The distance formula is calculated for the components of the target pixel using the improved distance formula Among them, H, S, V represent the H, S, V components in the HSV space respectively; H0, S0, V0 represent the target components H0, S0, V0 of the target pixel point.
4. A highway guardrail extraction method according to claim 1, characterized in that: The specific steps of morphological reconstruction are as follows: The complement of the obtained binary image is recorded as I c : And form a marked image with a boundary of 1, denoted as F: Assuming that the structural element to be reconstructed is B, the following operations are performed to remove isolated noise points in the image:
5. The highway guardrail extraction method according to claim 1, characterized in that: The specific steps of similarity clustering are as follows: There are N pixels in the synthetic image. For each pixel, an initialization label L(p) = -1 and an initialization distance d(p) = ∞ are given. The K blocks divided into required areas will be initialized with the seed point of each block as the cluster center: M k =[l k ,a k ,b k ,x k ,y k ,t k ]; The distance between each cluster center is Then calculate the cluster center M k The distance from the surrounding 2S×2S pixels to the cluster center: If D k <d(p) then update d(p) = D k ,L(p)=k, and use C k Denote the set of pixels with label L(p) = k, forming a new cluster center: Among them, |C k | is the set C k The number of pixels; and continuously iteratively calculate the new C k and residual error, and the iteration stops when the residual error is less than the threshold.