Street view image-based sidewalk width estimation method and system

Abstract

The invention provides a street view image-based sidewalk width estimation method and system. The method comprises the steps of S1, acquiring a 2D image of a street view; s2, performing semantic segmentation on the 2D image to generate a sidewalk area mask; s3, performing target detection on the 2D image to obtain a pedestrian target, determining a scale reference object based on the pedestrian target, and taking the pixel width of the scale reference object as the reference pixel width; s4, based on a semantic segmentation result, extracting contour line segments of the road from the 2D image, estimating vanishing points of the image, and calculating a shooting angle of the 2D image; s5, based on the sidewalk area mask, extracting an original sidewalk pixel width at a sidewalk position corresponding to the scale reference object; and geometric correction is performed on the original sidewalk pixel width and the preset human body real width by using the shooting angle, and the real width of the sidewalk is calculated based on the reference pixel width and the corrected human body real width represented by the reference pixel width.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and intelligent transportation, and in particular to a sidewalk width estimation method and system based on street view images. BACKGROUND

[0002] With the continuous advancement of smart cities, sidewalk width as an important parameter of urban infrastructure plays an increasingly important role in urban planning, building a safe and efficient intelligent transportation system, etc. Accurate acquisition of sidewalk width information in a large area is an important basis for city managers to develop scientific urban planning schemes, optimize traffic design, and evaluate the quality of the walking environment. For example, in the case of MiC (Modular Integrated Construction), a new green construction mode, the large-size MiC module transport vehicles in the city road are easy to have a significant impact on the safety of pedestrians on narrow sidewalks. Based on the distribution characteristics of the sidewalk width in the whole city, the planning of the MiC transport route can be effectively supported to avoid high-risk road sections, balance the transportation demand and the use safety of the pedestrian space in urban construction, and echo the development concept of "people-oriented" in the construction of smart cities.

[0003] Traditional sidewalk width measurement methods mainly rely on manual field measurement, which is low in efficiency and high in cost. In recent years, with the development of sensor technology, more and more researches have turned to using laser radars to accurately model the surrounding environment in three dimensions. However, on the one hand, the mobile measurement system based on laser radars is expensive, and on the other hand, it is necessary to collect data by oneself to realize large-scale city modeling, which is not cost-effective.

[0004] Therefore, there is an urgent need to develop a new technology that can accurately and quickly measure the sidewalk width based on existing street view images to meet the actual needs of acquiring sidewalk width information in large-scale urban areas.

[0005] It should be noted that the information disclosed in the above background section is only for understanding the background of the present application, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0006] The present application provides a sidewalk width estimation method and system based on street view images, which can accurately and quickly measure the sidewalk width.

[0007] In a first aspect, a sidewalk width estimation method based on street view images is provided, comprising the following steps: S1, obtaining a 2D image of a street view; S2, performing semantic segmentation on the 2D image to generate a sidewalk region mask; S3, target detection is performed on the 2D image to obtain a pedestrian target, a scale reference object is determined based on the pedestrian target, and a pixel width of the scale reference object is obtained as a reference pixel width; S4, based on the semantic segmentation result obtained in step S2, a contour line segment of a road is extracted from the 2D image, a vanishing point of the image is estimated based on the extracted line segment, and a shooting angle of the 2D image is calculated based on the estimated vanishing point; S5, based on the sidewalk region mask, an original sidewalk pixel width is extracted at a position of the sidewalk corresponding to the scale reference object; the original sidewalk pixel width and a preset human real width are geometrically corrected using the shooting angle to obtain a corrected sidewalk pixel width and a corrected human real width; and a real width of the sidewalk is calculated based on the reference pixel width and the corrected human real width represented thereby.

[0008] In a second aspect, a sidewalk width estimation system based on a street view image is provided, comprising: a processor; a memory having a computer program stored thereon and executable on the processor, wherein the processor implements the sidewalk width estimation method based on the street view image according to the first aspect when executing the computer program.

[0009] In a third aspect, a vehicle transportation route planning method is provided, comprising: obtaining sidewalk width information of a plurality of roads in a target region, wherein the sidewalk width information is estimated by using the method according to the first aspect; identifying a road segment with a passing risk based on the obtained sidewalk width information and a size parameter of a vehicle to be transported; and planning a transportation route to avoid the road segment with the passing risk.

[0010] The sidewalk width estimation method based on a street view image provided by the present application automatically and accurately estimates the physical width of a sidewalk based on widely available street view panoramic images, accurately locates the sidewalk region through semantic segmentation, and provides a reliable basis for geometric measurement in combination with a pedestrian scale reference object; the shooting angle is solved through vanishing point estimation based on road semantic features, and the preset human real width and the sidewalk pixel width are geometrically corrected in cooperation using the shooting angle, which finally collectively overcomes the geometric deviation caused by the variable perspective of the street view image, and realizes accurate conversion from the pixel dimension to the physical dimension.

[0011] Further advantages of the present application include the following: 1. The present application uses pedestrians as natural scale reference objects, avoiding the complex camera calibration process in traditional methods. Further, the average human width constant based on human engineering is used to preset the real width of the human body, and the real width of the human body is geometrically corrected according to the shooting angle, providing a reliable scale reference, so that the measurement result has good consistency and accuracy.

[0012] 2. This invention leverages the powerful generalization ability of deep learning models and targeted algorithm optimizations to maintain stable and reliable performance in real-world applications. Furthermore, the intelligent pedestrian screening algorithm automatically excludes pedestrians located at image edges and those occluded by other objects, ensuring the selection of the optimal scale reference object.

[0013] 3. This invention uses semantic segmentation results to quickly estimate vanishing points, achieving accurate and reliable results. Furthermore, based on the semantic results, it focuses only on edge extraction of vehicle, road, and pedestrian areas, and excludes edge interference from vehicle areas, thus achieving faster, more accurate, and more reliable vanishing point estimation.

[0014] 4. Due to the special nature of street view images, the images in this invention have different shooting angles (and are not necessarily taken along the road direction). This invention estimates the shooting angle based on the vanishing point and field of view, and performs targeted correction on the sidewalk width according to the shooting angle, further improving the estimation accuracy.

[0015] 5. This invention requires no additional sensors; the entire estimation process only requires street view images as input. The data used is readily available and can be quickly applied to a wide range of cities, making this technology a promising prospect for commercial application. Attached Figure Description

[0016] Figure 1 This is a flowchart of a sidewalk width estimation method based on street view images according to the present invention.

[0017] Figure 2 This is a breakdown of the panoramic image.

[0018] Figure 3 This is a binary mask for the semantic segmentation result and the pedestrian area.

[0019] Figure 4a It is the vehicle road mask image in step S4.

[0020] Figure 4b It is the edge image extracted in step S4.

[0021] Figure 4c It is the line segment image fitted in step S4.

[0022] Figure 4d It is the image obtained by line segment filtering based on the surrounding semantic situation in step S4.

[0023] Figure 4e It is the final vanishing point image obtained in step S4.

[0024] Figure 5a andFigure 5b These are images showing the selection of reference pedestrians and the extraction of sidewalk pixel widths, respectively.

[0025] Figure 6 This is the estimation result of the method of the present invention.

[0026] Figure 7 This is a diagram showing the sidewalk width estimation error of the method of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0028] The research objective of this invention is to avoid the time-consuming and labor-intensive data collection process by using the already abundant street view data (usually 360° panoramic images). Figure 1 A flowchart illustrating a sidewalk width estimation method based on street view images provided by an embodiment of the present invention is shown. The sidewalk width estimation method based on street view images in this embodiment of the invention is applied to devices such as computers and servers. See also... Figure 1 The method includes the following steps: Step S1: Obtain a 2D image of the street view.

[0029] Specifically, in step S1, the street view panoramic image can be processed to obtain multiple 2D images with predetermined field of view angles. There are two strategies for selecting 2D images for subsequent processing: one is to select 2D images along the road direction for subsequent processing (e.g., the front and rear directions of the street view image acquisition vehicle, i.e., 2D images with a field of view angle of 90° or 270°); the other is to process multiple 2D images with predetermined field of view angles. Regardless of which strategy is selected, when obtaining the actual sidewalk width, a value within a predetermined range is selected as the actual sidewalk width, for example, a value within the range of 0.5m to 5m is selected as the actual sidewalk width. Values ​​outside this range are considered outliers and are filtered out.

[0030] Specifically, in step S1, one of the instances is... Figure 2 As shown, the street view panoramic image is split into four 2D images with field of view angles of 0°, 90°, 180° and 270°. In this example, the 2D image with a field of view angle of 90° is selected for subsequent processing.

[0031] S2. Perform semantic segmentation on the 2D image to generate a sidewalk area mask.

[0032] In step S2, one example involves using the Segformer semantic segmentation model, based on the Transformer architecture, to perform pixel-level semantic analysis on a 2D image. Specifically, the Segformer-B3 model, pre-trained on the Cityscapes dataset, is used. This model can recognize 19 categories, including vehicles, roads, sidewalks, buildings, vehicles, and pedestrians. The semantic segmentation process includes: first, using the SegformerImageProcessor function to extract features from the input 2D image, generating tensor format data suitable for model input; then, inputting the processed data into the SegformerForSemanticSegmentation model for inference computation; finally, using bilinear interpolation to restore the output logits (original predicted values) to the original image size through upsampling, and using argmax operation to obtain the final semantic category prediction result for each pixel.

[0033] like Figure 3 As shown, (a) is the original image, i.e., the 2D image with a field of view of 90° obtained in step S1, as shown. Figure 3 As shown in Figure (b), the semantic segmentation results are output as pixel-level label maps, where the label category ID for the vehicle road region is 0 (displayed in dark magenta, with RGB values ​​of [128, 64, 128]), and the label category ID for the pedestrian walkway region is 1 (displayed in bright magenta, with RGB values ​​of [244, 35, 232]). For ease of subsequent processing, this invention extracts the pedestrian walkway region as a binary mask, where the pedestrian walkway pixel value is 255, and the pixel values ​​for other regions are 0, as shown below. Figure 3 As shown in Figure (c), the white area is the sidewalk area, and the mask of this sidewalk area (specifically a binary mask) will serve as the basis for subsequent extraction of the sidewalk width.

[0034] S3. Perform target detection on the 2D image to obtain pedestrian targets, determine a scale reference object based on the pedestrian targets, and obtain the pixel width of the scale reference object as the reference pixel width.

[0035] Specifically, in step S3, target detection is performed on the output of step S1, and the detected pedestrian targets are intelligently filtered to obtain a reference pixel width. One example is using a target detection model (such as YOLOv11) to detect targets in a 2D image. YOLOv11, as one of the most advanced real-time target detection algorithms, can provide fast processing speed while ensuring detection accuracy. The detection process outputs bounding box coordinates, confidence scores, and category labels for various targets. Semantic segmentation in step S2 can also detect targets such as pedestrians, but its accuracy is slightly lower than that of the YOLOv11 model used in this step. If further improvement in running efficiency is required, or if lower accuracy is required, the target detection process in this step can be replaced by step S2.

[0036] To ensure the selection of the most suitable pedestrian as a scale reference, this invention designs an intelligent pedestrian target screening mechanism. This intelligent screening mechanism selects pedestrians that meet preset conditions as scale reference objects, including the following steps: S31. Perform edge filtering to exclude pedestrians located in the edge area of ​​the image; and perform occlusion detection to exclude pedestrians occluded by objects of a preset category (such as vehicles), thereby filtering out valid pedestrians.

[0037] Specifically, in one instance of edge filtering to exclude pedestrians located in image edge regions, the left edge threshold is defined as 5% of the image width, the right edge threshold as 95% of the image width, the top edge threshold as 5% of the image length, and the bottom edge threshold as 95% of the image length. If the left boundary of a pedestrian's bounding box is less than the left edge threshold or the right boundary is greater than the right edge threshold, or the top boundary is less than the top edge threshold or the bottom boundary is greater than the bottom edge threshold, then the pedestrian is marked as invalid. This is because pedestrians in edge regions are often partially truncated, making it impossible to provide accurate width information. In another instance of occlusion detection to exclude pedestrians occluded by preset category objects (such as vehicles), the method is to check the semantic segmentation results within a certain range below each pedestrian's bounding box. If this range contains pixels representing vehicle categories such as private cars, buses, and trucks, then the pedestrian is considered occluded. The height range for occlusion detection can be set to 10 pixels or the distance from the bottom edge of the pedestrian's bounding box to the bottom edge of the image, whichever is smaller.

[0038] S32. Among all the selected valid pedestrians, select the bottom edge of the bounding box. y The pedestrian with the largest coordinate value is used as the scale reference object, and its pixel width is used as the reference pixel width. The pixel width is calculated by subtracting the left boundary coordinate from the right boundary coordinate of the bounding box of the reference pedestrian. y The largest coordinate means that the pedestrian is at the lowest position in the image, closest to the camera, and provides the most accurate scale reference. If multiple pedestrians have the same largest coordinate...y The coordinates can be used to calculate the average width of these pedestrian bounding boxes as a reference pixel width.

[0039] S4. Based on the semantic segmentation results of the vehicle road area and sidewalk area obtained in step S2, extract the contour line segments of the road (including the sidewalk) from the 2D image, estimate the vanishing point of the image based on the extracted line segments, and calculate the shooting angle of the 2D image based on the estimated vanishing point.

[0040] like Figures 4a-4e As shown, in step S4, the outputs of steps S1 and S2 are used to perform fast vanishing point estimation based on semantic segmentation of the road region and shooting angle estimation based on vanishing point and field of view. Specifically, this includes the following steps: S41. Extract the vehicle road region and sidewalk region from the semantic segmentation results.

[0041] S42. Perform edge detection and line segment fitting on the extracted vehicle road area and sidewalk area respectively, merge the fitted line segments of the vehicle road area and sidewalk area, and filter out the horizontal and vertical line segments to obtain the initial line segment set.

[0042] Specifically, firstly, the vehicle and road regions are extracted from the semantic segmentation results, such as... Figure 4a As shown, this is a vehicle road mask image; the white area represents the vehicle road region in semantic segmentation. Subsequently, an edge detection algorithm was used to extract the image edges, as shown below. Figure 4b As shown, the white lines represent the extracted edges. One example is using the Sobel operator to extract edge information from an image, calculating the horizontal direction (…). x Direction) and vertical direction ( y The gradient in the horizontal direction is calculated, with a kernel size (ksize) set to 3. These are then combined to calculate the magnitudes of the horizontal and vertical gradients as edge magnitudes, with a threshold of 50 to obtain sharp edges. Then, an algorithm based on a line segment detector (LSD) is used to extract line segments from the edge image, as shown in the following figure. Figure 4c As shown (the red lines are line segments based on edge fitting), each extracted line segment is represented by its start and end coordinates. This method has sub-pixel accuracy and is suitable for extracting directional and continuous line segments from edge maps. Its parameters are set to the standard fine-tuning mode LSD_REFINE_STD.

[0043] Next, repeat the above steps for the pedestrian area, merge the fitted line segments of the vehicle road area and the pedestrian area, and filter out the horizontal and vertical line segments to obtain the initial set of line segments.

[0044] S43. Verify and filter the line segments in the initial line segment set to obtain a valid line segment set.

[0045] In step S43, each line segment in the initial line segment set is cross-validated and filtered with the original image to obtain a set of valid line segments, including: First, for each fitted line segment, check if it covers a brighter (high grayscale value) area in the original image. If so, it is considered a valid line segment. Specifically, the original image is first converted to grayscale. Then, for each fitted line segment, a corresponding line segment mask region is created on the grayscale image. The thickness of the mask drawing line can be set to 3 pixels. Subsequently, the set of pixel grayscale values ​​corresponding to the line segment mask region is extracted. A preset intensity threshold can be set to 30. If the average value of the grayscale value set is greater than this value, the line segment is retained as a valid line segment.

[0046] Subsequently, further segment filtering is performed based on the semantic categories surrounding the segment to improve the accuracy of subsequent vanishing point estimation, such as... Figure 4d The image shown is a line segment filtering result based on surrounding semantic conditions. The percentage next to each line segment represents the proportion of prohibited semantic categories. Red lines indicate filtered-out segments, while green lines indicate retained segments. Although the proportion of prohibited semantic categories in the middle of the image exceeds the threshold, these segments are skipped and retained because multiple semantic categories are assigned to the same pixel in this area, indicating low reliability of the semantic segmentation result. Specifically, a line segment mask is generated for each line segment, and a corresponding "line segment region mask" is generated according to a preset expansion radius (e.g., 5 pixels). Then, the number of semantic categories in each line segment region is counted. If the number of categories is greater than or equal to the threshold (e.g., 5), the semantic segmentation result in that region is considered unreliable, and the line segment is skipped. Otherwise, subsequent processing is performed. Subsequent processing includes setting prohibited semantic categories. For example, to filter out the edges formed by roads and vehicles, the prohibited semantic categories can be set to 13 (private car), 14 (truck), 15 (bus), and 16 (train). Then, the pixels of the corresponding prohibited semantic categories are extracted to generate a binary mask, and morphological dilation is performed (e.g., dilution by 1 pixel) to increase stability. Within the line segment region corresponding to each line segment, the ratio of the number of prohibited semantic category pixels to the total number of pixels is calculated to obtain the prohibited semantic category percentage. If the prohibited semantic category percentage is less than a preset threshold (e.g., 30%), the line segment is retained; otherwise, it is filtered out.

[0047] S44. Based on the set of valid line segments, calculate the intersection points of the line segments as candidates for the extinction point, and after eliminating the outliers, calculate the arithmetic mean of the remaining candidate points as the final extinction point.

[0048] In step S44, the intersections of the valid line segments extracted in the previous step are calculated as candidates for the vanishing point. The center points of these candidate points are calculated, and the distances between the candidate points and the center points are calculated. A threshold of 100 is set; candidate points exceeding this threshold are considered outliers and eliminated. The arithmetic mean of the final candidate point set is calculated as the final estimated vanishing point. Figure 4e As shown in the figure, the red dots are the final calculated vanishing points, the red lines are line segments fitted based on the vehicle road area and the pedestrian area, the blue dots are the inliers of the line segment intersections, and the blue crosses are the outliers of the line segment intersections.

[0049] S45. Based on the final vanishing point in the image coordinate system x The shooting angle is calculated using formula (1) based on the coordinates on the axis, image width, and field of view. Ψ : (1) in Vx The final vanishing point is in the image coordinate system. x Coordinates on the axis W It is the width of the image. α It is the field of view. In this example, W It's 400. α The angle is 90°, and the vanishing point coordinates are (150.83, 207.72). Using the formula above, the shooting angle is calculated. Ψ The angle is 0.38 rad. In this invention, if the road is straight and the photo is taken along the road, the shooting angle is 0 rad.

[0050] S5. Based on the pedestrian area mask (a binary mask in this example), extract the original pedestrian pixel width at the foot position of the scale reference object; use the shooting angle to perform geometric correction on the original pedestrian pixel width and the preset human body true width to obtain the corrected pedestrian pixel width and the corrected human body true width; calculate the true width of the pedestrian side based on the reference pixel width and the corrected human body true width it represents.

[0051] In step S5, based on the outputs of steps S2, S3, and S4, the width of the sidewalk is accurately estimated, including: S51. Using the y-coordinate of the foot (i.e. the bottom edge of the pedestrian bounding box) of the scale reference object as a reference, extract the pedestrian pixel distribution at the y-coordinate position from the pedestrian area mask. S52. Determine continuous segments of the sidewalk based on the sidewalk pixel distribution, and select the continuous segment of the sidewalk that is closest to the scale reference object; S53. Calculate the pixel width of the selected continuous segment of the sidewalk as the original sidewalk pixel width. wps_r ; S54, using the aforementioned shooting angle Ψ The original sidewalk pixel width is obtained by formula (2). wps_r The correction is performed to obtain the corrected sidewalk pixel width. wps: (2); S55, The actual width of the human body is preset based on the average width constant of the human body. a Perform the same correction as in step S54 to obtain the corrected true width of the human body. a cos Ψ ; S56. Calculate the actual width of the sidewalk according to formula (3). wrs ,in wpp It is the reference pixel width: (3).

[0052] Specifically, in step S5, the reference pixel width wpp This is obtained from step S3. Based on the pedestrian area mask obtained in step S2, the pixel width of the pedestrian walkway is extracted at the foot coordinates of the reference pedestrian, where the reference pedestrian is also obtained from step S3, such as... Figures 5a-5b The figures shown are the reference pedestrian selection and the sidewalk pixel width extraction, respectively. Figure 5a In the diagram, the yellow boxes represent pedestrian targets detected in step S3, while the red boxes represent reference pedestrians determined after intelligent filtering of pedestrian targets. Figure 5b In the diagram, the red dashed line represents the horizontal line formed by the y-coordinates of the reference pedestrian's feet, and the green line segment represents the extracted original sidewalk pixel width. Specifically, it is based on the bottom edge of the reference pedestrian bounding box. y Using coordinates as a reference, the pixel distribution of the sidewalk mask on the horizontal line is extracted. Since the sidewalk area may be discontinuous (e.g., obscured by vehicles, obstacles, etc.), this invention employs a segmented continuous region recognition algorithm. First, the pixel distribution of all sidewalk pixels on the horizontal line is identified. x Coordinates, and then according to x The continuity of coordinates is segmented; if two adjacent pixels... x The coordinate difference is greater than 1.5. wpp If a pixel value is considered to have a breakpoint, it is divided into different continuous segments. Among these continuous segments, the pedestrian crossing segment closest to the reference pedestrian is selected as the original pedestrian crossing pixel width. wps_r The distance here is defined as the distance from the segment center point to the pedestrian bounding box center point in pixel coordinates. x Manhattan distance on the axis. Selected original sidewalk pixel width. wps_r The calculation method is based on the maximum value of this segment.x Coordinates minus minimum x coordinate.

[0053] To further eliminate the systematic influence of the camera shooting angle on the estimation results, this invention has implemented a correction. Based on the shooting angle calculated in step S4... Ψ The original sidewalk pixel width is calculated using the following formula. wps_r Perform the corresponding geometric corrections: (2) wps This is the corrected pixel width of the sidewalk. The same correction should also be applied to the preset actual human body width. The final formula for calculating the actual sidewalk width is as follows: (3) in a The preset human body width is derived from the average human body width constant. There are two main reasons for choosing human body width as a reference: firstly, pedestrians are widely present in street scenes; and secondly, width is more stable than height. Based on ergonomic research data, the aforementioned average human body width constant... a This is a predefined statistical value, typically ranging from 0.45 meters to 0.55 meters. The average shoulder width of an adult is approximately 0.52 meters. a The value can be configured and adjusted according to the characteristics of pedestrian groups in the target area (such as areas around schools, commercial areas, etc.). In this embodiment of the invention, 0.52 meters is used as the preset actual width of the human body for estimation, and one estimation result is as follows: Figure 6 As shown in the figure, the red dot is the estimated vanishing point, the red box is the reference pedestrian, the green line segment is the estimated sidewalk, and the white text on the figure is the estimated shooting angle (0.2304 rad) and the sidewalk width (3.795 m).

[0054] This invention combines deep learning-based semantic segmentation and object detection, fast vanishing point estimation of road regions based on semantic segmentation, and a correction method based on shooting angle to achieve accurate and reliable estimation of sidewalk width. Compared to traditional methods, this invention requires no additional sensor hardware costs and is suitable for large-scale sidewalk width estimation.

[0055] An application example of this invention in the field of intelligent transportation is as follows: using this method to estimate the width of sidewalks throughout the city, and based on their distribution characteristics, rationally planning the routes of large transport trucks using MiC (Modular Integrated Construction) modules, avoiding high-risk road sections, balancing the transportation needs of urban construction with the safety of pedestrian space use, and echoing the "people-oriented" development concept in the construction of smart cities.

[0056] In a specific example, the vehicle transportation route planning method includes: obtaining pedestrian width information of multiple roads within a target area, wherein the pedestrian width information is estimated using the method described in this invention; identifying road sections with traffic risks based on the obtained pedestrian width information and the size parameters of the vehicle to be transported; and planning a transportation route to avoid the road sections with traffic risks.

[0057] In MiC (Military-Oriented Community) transportation scenarios, the safe distance between transport vehicles and existing traffic facilities (such as road shoulders and streetlights) is typically required to be at least 15 centimeters. To ensure the safety of urban pedestrian spaces, the distance between MiC transport vehicles and pedestrians should also not be less than the aforementioned safe distance. According to test results, the error of the pedestrian walkway width estimation method based on a preset actual human body width proposed in this invention is basically proportional to the actual width of the pedestrian walkway, with an error ratio of approximately 10% to 20%. (The potential error of this invention mainly stems from the natural differences in human body width. This error can be mitigated to some extent by using multiple street view images from the same road to process multiple estimated pedestrian walkway widths using the algorithm in this invention, and then averaging these values ​​(because different street view images generally use different reference pedestrians). Furthermore, a safety factor can be introduced in subsequent applications (such as route planning), for example, reserving a redundancy margin of 0.2 based on the upper bound of the error, thereby controlling the impact of the estimation error on actual decision-making within an acceptable range.) Figure 7 As shown, the estimated value is usually smaller than the actual width. This smaller estimation characteristic has certain advantages in MiC transportation route planning, because distance design usually needs to leave redundancy space. The estimation results of this invention can improve the safety margin of route evaluation to a certain extent, thus better meeting the actual engineering needs.

[0058] A specific embodiment of the present invention also provides a sidewalk width estimation system based on street view images, comprising: a processor; and a memory storing a computer program that can run on the processor, wherein the processor executes the computer program to implement the sidewalk width estimation method based on street view images.

[0059] A specific embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned method for estimating sidewalk width based on street view images.

[0060] The above description provides a further detailed explanation of the present invention in conjunction with specific / preferred embodiments, and it should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various substitutions or modifications can be made to these described embodiments without departing from the concept of the present invention, and all such substitutions or modifications should be considered within the scope of protection of the present invention. In the description of this specification, the reference to terms such as "an embodiment," "some embodiments," "preferred embodiment," "example," "specific example," or "some examples," etc., indicates that the specific features, structures, materials, or characteristics described in connection with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples. Although the embodiments of the present invention and their advantages have been described in detail, it should be understood that various changes, substitutions, and modifications can be made herein without departing from the scope of protection of the patent application.

Claims

1. A sidewalk width estimation method based on street view images, characterized by, The method comprises the following steps: S1, acquiring a 2D image of a street view; S2, performing semantic segmentation on the 2D image to generate a sidewalk region mask; S3, performing target detection on the 2D image to obtain a pedestrian target, determining a scale reference object based on the pedestrian target, and obtaining a pixel width of the scale reference object as a reference pixel width; S4, based on the semantic segmentation result obtained in step S2, extracting a contour line segment of a road from the 2D image, estimating a vanishing point of the image based on the extracted line segment, and calculating a shooting angle of the 2D image based on the estimated vanishing point; S5, based on the sidewalk region mask, extracting an original sidewalk pixel width at a sidewalk position corresponding to the scale reference object; performing geometric correction on the original sidewalk pixel width and a preset human real width by using the shooting angle to obtain a corrected sidewalk pixel width and a corrected human real width; based on the reference pixel width and the corrected human real width represented thereby, calculating a real width of the sidewalk.

2. The sidewalk width estimation method based on street view images according to claim 1, wherein, In step S3, determining a scale reference object based on the pedestrian target comprises: S31, excluding pedestrians located in an edge region of the image and pedestrians blocked by a preset category object, and screening out valid pedestrians; S32, selecting a bottom edge of the bounding box from the filtered effective pedestrians y The pedestrian with the largest coordinate value is taken as the scale reference object. In step S31, excluding pedestrians located in an edge region of the image comprises: defining an image left edge threshold as a first predetermined value of the image width, a right edge threshold as a second predetermined value of the image width, an upper edge threshold as a third predetermined value of the image width, and a lower edge threshold as a fourth predetermined value of the image width; when a left boundary of a pedestrian bounding box is less than the left edge threshold or a right boundary is greater than the right edge threshold, or when an upper boundary of the pedestrian bounding box is less than the upper edge threshold or a lower boundary is greater than the lower edge threshold, the pedestrian is excluded; In step S31, excluding pedestrians blocked by a preset category object comprises: determining a range for occlusion detection according to a smaller value of a distance from a bottom edge of a pedestrian bounding box to a bottom edge of the image and a preset pixel value; when a pixel of a preset category object is included in the range, it is considered that the pedestrian is blocked.

3. The method according to any one of claims 1-2, characterized in that, Step S4 comprises: S41, extracting a vehicle road region and a sidewalk region in the semantic segmentation result; S42, performing edge detection and line segment fitting on the extracted vehicle road region and sidewalk region respectively, merging the line segments fitted for the vehicle road region and the sidewalk region, and filtering out horizontal line segments and vertical line segments to obtain an initial line segment set; S43, verifying and screening the line segments in the initial line segment set to obtain an effective line segment set; wherein the verification and screening comprise: first, screening based on the gray intensity of the line segments on the original gray image, and retaining the line segments covering high gray value pixels; then, performing line segment screening on the line segments covering high gray value pixels based on the semantic categories of the surrounding regions of the line segments, and filtering out the line segments whose surrounding region prohibited semantic category proportion exceeds a preset threshold; S44, calculating the intersection points of the line segments as vanishing point candidates based on the effective line segment set, and calculating the arithmetic mean of the remaining candidate points as the final vanishing point after eliminating outliers. S45. determining the shooting angle based on the final vanishing point in the image coordinate system x the image width and the field of view angle, the shooting angle is calculated by formula (1): (1) wherein Vx is the shooting angle, ​ is the coordinate of the final vanishing point in the image coordinate system x axis, W is the width of the image, α is the field of view angle.

4. The method of claim 3, wherein: In step S43, the filtering based on the gray intensity of the line segment on the original gray image to retain the line segment covering the high gray value pixel includes: first, performing gray processing on the original image, then creating a corresponding line segment mask area on the gray image for each fitted line segment, and then extracting the gray value set of the pixels corresponding to the line segment mask area and setting a preset intensity threshold, the line segment with an average value of the gray value set greater than the intensity threshold is retained as an effective line segment; In step S43, the line segment filtering based on the semantic category of the line segment peripheral region to filter out the line segment whose peripheral region prohibited semantic category proportion exceeds the preset threshold includes: first, judging the reliability of the semantic segmentation result of the line segment peripheral region, if the number of semantic categories in the region is higher than the preset value, it is judged that the reliability is low, and the filtering of the line segment is skipped, otherwise, the prohibited semantic category is set, and the semantic situation of the periphery of each fitted line segment is checked, if the prohibited semantic category proportion in the line segment peripheral region is less than the preset threshold, the line segment is retained, otherwise, it is filtered out.

5. The method according to any one of claims 1-2, characterized in that, Step S5 includes: S51, taking the bottom edge y coordinate of the bounding box of the scale reference object as a reference, extracting the sidewalk pixel distribution at the y coordinate position from the sidewalk region mask; S52, determining a sidewalk continuous segment based on the sidewalk pixel distribution, and selecting a sidewalk continuous segment closest to the scale reference object; S53, calculate the pixel width of the selected sidewalk segment as the original sidewalk pixel width wps_r ; S54, using the aforementioned shooting angle ψ The original sidewalk pixel width is obtained by formula (2). wps_r The correction is performed to obtain the corrected sidewalk pixel width. wps: (2)； S55, correcting the human real width based on the human average width constant a The same correction as step S54 is performed to obtain the corrected human real width a cos ψ ; S56, calculate the real width of the sidewalk according to formula (3) wrs wherein wpp is the reference pixel width: (3)。 6. The method of claim 5, wherein, In step S52, the method of determining the sidewalk continuous segment includes: setting a breakpoint threshold based on the reference pixel width, and dividing the continuous segment according to the comparison between the coordinate difference of adjacent sidewalk pixels and the breakpoint threshold.

7. The method of claim 1, wherein, In step S1, the 2D image of the street view is obtained by splitting a street view panorama image.

8. The method of claim 1, wherein, The 2D image is subjected to semantic segmentation and object detection using a deep learning model, which is a Segformer model based on a Transformer architecture; Or in step S2, the 2D image is subjected to semantic segmentation using a first deep learning model, which is a Segformer model based on a Transformer architecture, and in step S3, the 2D image is subjected to object detection using a second deep learning model, which is an object detection model.

9. A sidewalk width estimation system based on street view images, characterized by, Comprise: a processor; a memory having a computer program stored thereon and executable on the processor, wherein the processor executes the computer program to implement the method for estimating the sidewalk width based on the street view image according to any one of claims 1-9.

10. A vehicle transportation route planning method characterized by comprising: Comprise: obtaining sidewalk width information of multiple roads in a target area, wherein the sidewalk width information is estimated by the method according to any one of claims 1-8; based on the obtained sidewalk width information and the size parameters of the vehicle to be transported, identifying road segments with traffic risks; planning a transportation route to avoid the road segments with traffic risks.

Images