Image-based curb recognition method and system

Abstract

The application discloses an image-based road edge recognition method and system, the method comprising: acquiring a road scene image of a current frame, inputting the road scene image into a road semantic segmentation model, outputting a road semantic label map, obtaining an initial road edge observation point sequence and a corresponding observation confidence evaluation value of the current frame and a lane line geometric parameter set by using a road observation mechanism; determining a reference lane line from the lane line geometric parameter set and identifying a spatial constraint mode between the initial road edge observation point sequence and the reference lane line; constructing a time sequence feature vector based on a spatial constraint mode of continuous multiple frames within a preset window, evaluating a comprehensive confidence of a spatial constraint relationship of the current frame; and generating a discrete coordinate point set of a final road edge curve according to a comparison result of the observation confidence and the comprehensive confidence by using a preset routing strategy. Thus, high-reliability and high-precision road edge recognition is achieved.

Description

Technical Field

[0001] This invention relates to the field of visual perception technology for autonomous driving, and in particular to an image-based roadside recognition method and system. Background Technology

[0002] For autonomous vehicles, especially specialized vehicles performing tasks such as automatic edge cleaning and line-following patrols, accurate and robust environmental perception capabilities are crucial for ensuring operational safety and efficiency. The curb, as the physical boundary between road and off-road areas, is a key perception target guiding these vehicles in edge-following driving or operations.

[0003] Currently, vision-based roadside recognition methods have become the mainstream technology in this field. A typical approach involves the following steps: acquiring road images using a camera mounted on a vehicle; using deep learning-based semantic segmentation models (such as FCN and DeepLab) to identify feasible road regions in the images, thus initially segmenting the pixel locations of the roadside; subsequently, using post-processing algorithms (such as connected component analysis, edge point extraction, and curve fitting) to extract the geometric representation of the roadside (such as a set of points or a curve) from the segmentation mask. To improve output stability, existing technologies typically introduce temporal filtering algorithms (such as Kalman filters and sliding window averaging) to smooth the roadside geometry identified frame by frame, suppressing noise and jitter in single-frame images.

[0004] However, the aforementioned existing technologies still have significant limitations when dealing with complex real-world road scenarios: they heavily rely on the visibility of the roadside itself. When the roadside is temporarily or severely obscured (such as by large vehicles, dense pedestrians, or temporary parking objects), the segmentation model cannot obtain effective roadside pixel information, resulting in a sharp decline in observation quality. Summary of the Invention

[0005] This application provides an image-based roadside recognition method that extracts roadside and lane line information, uses a temporal model to evaluate the reliability of their spatial relationship, and adaptively selects the optimal generation strategy based on a dual judgment of observation quality and relationship reliability. Thus, even when the roadside is severely occluded, it can still infer a reasonable virtual roadside based on reliable lane line information, effectively solving the problems of continuity and robustness of roadside perception in complex scenarios.

[0006] This application provides an image-based road edge recognition method, including:

[0007] S101: Obtain the road scene image of the current frame, input it into the pre-trained road semantic segmentation model, output the road semantic label map, and use the preset road observation mechanism to obtain the initial roadside observation point sequence of the current frame and its corresponding observation confidence evaluation value, as well as the set of lane line geometric parameters.

[0008] S102, determine the reference lane line from the set of lane line geometric parameters, and identify the spatial constraint pattern between the initial roadside observation point sequence and the reference lane line. The spatial constraint pattern includes at least the parallel constraint pattern and the convergence / separation constraint pattern.

[0009] S103, Evaluate the comprehensive confidence of the spatial constraint relationship of the current frame based on the time series feature vector constructed from the spatial constraint pattern of multiple consecutive frames within a preset window;

[0010] S104. Based on the comparison between the observed confidence level and the overall confidence level, a set of discretized coordinate points for the final roadside curve is generated using a preset routing strategy.

[0011] Preferably, the pixels in the road semantic label map are at least classified into drivable road areas, non-road areas, and lane line labels.

[0012] Preferably, the preset road observation mechanism specifically includes:

[0013] S201, boundary detection processing is performed on the semantic tag map to extract the boundary pixels between the drivable area and the non-road area. After coordinate transformation and denoising, the initial roadside observation point sequence of the current frame is formed, and the observation confidence evaluation value corresponding to the sequence is calculated. The observation confidence evaluation value is obtained by extracting boundary integrity, point sequence distribution density, and boundary contrast based on the initial roadside observation point sequence.

[0014] S202, cluster and parametric curve fitting are performed on the lane line category pixels in the semantic label map to obtain the set of lane line geometric parameters for the current frame.

[0015] Preferably, the set of lane line geometric parameters includes the curve parameters of at least one lane line and its corresponding lane confidence level.

[0016] Preferably, the method for selecting the reference lane line is as follows:

[0017] For each lane line in the set of lane line parameters Calculate the average lateral distance from all points in the initial roadside observation point sequence to the lane line curve. Select to make The smallest lane line as a reference lane line .

[0018] Preferably, identifying the spatial constraint pattern between the initial roadside observation point sequence and the reference lane line specifically includes:

[0019] S301, based on reference lane lines Calculate the initial roadside observation point sequence Lateral offset sequence D relative to the reference lane line: along the reference lane line Within its preset effective longitudinal range Within, at preset fixed intervals Sample M longitudinal positions For each In the initial roadside observation point sequence Find its x-coordinate in the middle With the current sampling point The curb point with the smallest absolute difference is denoted as . and calculate The horizontal coordinate at this location , thus obtaining the horizontal offset sequence , For the corresponding curb point The horizontal coordinate, For reference, the lane lines are in the longitudinal position. The horizontal coordinate of the location;

[0020] S302, Based on the lateral offset sequence, determine the spatial constraint mode;

[0021] S303, calculate the spatial constraint strength index of the current frame based on the identified spatial constraint pattern.

[0022] Preferably, S302 specifically includes:

[0023] B1. Calculate the variance of the lateral offset sequence D. ,in The mean of the horizontally offset sequence, if ,and , The preset variance threshold, If the preset reasonable distance range between the curb and lane line is met, then the current frame's curb-lane line is determined to be in parallel mode. ;

[0024] B2. For all data pairs Perform a first-order linear regression to obtain the slope s, intercept b, and coefficient of determination for the fit. ,like and , The preset trend strength threshold, If a predefined threshold for trend significance is used, the slope sign is used to determine the convergence pattern: if s is less than 0, then it is a convergence pattern. If s is greater than 0, then it is in separation mode. .

[0025] Preferably, S103 specifically includes:

[0026] A time-series feature vector is constructed based on the spatial constraint pattern and spatial constraint strength index of multiple consecutive frames within a preset window. This vector is then input into a pre-trained temporal relationship evaluation model. The output represents the temporal stability score of the spatial constraint pattern and the anomaly confidence score, indicating whether the relationship features of the current frame are abnormal. In combination with the spatial constraint strength index, a comprehensive confidence score is calculated to characterize the credibility of the spatial constraint relationship of the current frame.

[0027] Preferably, the time series feature vector is set as follows:

[0028] C1. For each frame t within the preset window, construct the frame feature vector. Including spatial constraint modes One-hot coding and spatial constraint strength index Observation confidence assessment value Correlation geometric statistics; correlation geometric statistics include at least one or more of the mean and standard deviation of the horizontally offset series;

[0029] C2. Construct a temporal feature matrix from the frame feature vectors of L consecutive frames within the preset window. L is the number of frames, and H is the dimension of the frame feature vector.

[0030] This application also provides an image-based curb recognition system, including: an acquisition module, a recognition module, an evaluation module, and a curb generation module;

[0031] The acquisition module is used to acquire the road scene image of the current frame, input it into the pre-trained road semantic segmentation model, output the road semantic label map, and use the preset road observation mechanism to obtain the initial roadside observation point sequence of the current frame and its corresponding observation confidence evaluation value, as well as the set of lane line geometric parameters.

[0032] The identification module is used to determine the reference lane line from the set of lane line geometric parameters and identify the spatial constraint pattern between the initial roadside observation point sequence and the reference lane line. The spatial constraint pattern includes at least the parallel constraint pattern and the convergence / separation constraint pattern.

[0033] The evaluation module is used to evaluate the comprehensive confidence of the spatial constraint relationship of the current frame based on the time series feature vector constructed from the spatial constraint pattern of multiple consecutive frames within a preset window.

[0034] The curb generation module is used to generate a set of discretized coordinate points of the final curb curve based on the comparison results of the observation confidence and the comprehensive confidence, using a preset routing strategy.

[0035] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0036] By constructing a closed-loop framework from unified perception to intelligent decision-making, the continuity, robustness, and rationality of roadside boundary perception in complex and dynamic road scenarios of autonomous vehicles are significantly improved. First, a unified semantic segmentation model is used to jointly analyze road images, synchronously and consistently extracting the initial observation points of the roadside and the geometric parameters of the lane lines, laying a spatiotemporally aligned data foundation for subsequent analysis. Then, by quantifying and analyzing the spatial geometric relationship between the roadside point set and the reference lane lines, the complex scene structure is transformed into explicit topological constraint patterns (such as parallelism and convergence) and their strength indices. Based on this, a temporal prediction model is introduced to intelligently evaluate the historical evolution of the above constraint relationships, outputting their stability and anomaly degree, comprehensively forming an authoritative judgment on the credibility of the current structural relationships. Finally, the system... Based on the real-time calculated confidence scores of roadside observations and the comprehensive confidence scores of structural relationships, the system adaptively switches and executes three routing strategies: "observation-driven fitting," "structural relationship inference," and "weighted fusion." This ensures perception accuracy when roadside visibility is good, and when severe occlusion causes observation failure, it can rely on a deep understanding of the road topology to generate physically reasonable virtual roadsides based on highly reliable lane line information. This effectively solves the problems of perception interruption and misjudgment in extreme situations such as occlusion in traditional methods, providing stable and reliable perception input for safe and smooth roadside operation. Attached Figure Description

[0037] Figure 1 This is a schematic flowchart of an image-based road edge recognition method according to an embodiment of the present invention;

[0038] Figure 2 This is a structural block diagram of an embodiment of the present invention. Detailed Implementation

[0039] To facilitate understanding of the present invention, a more complete description of this application will be given below with reference to the accompanying drawings, which illustrate preferred embodiments of the invention. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to enable a more thorough and complete understanding of the disclosure of the present invention.

[0040] It should be noted that the terms "vertical," "horizontal," "up," "down," "left," "right," and similar expressions used in this article are for illustrative purposes only and do not represent the only possible implementation.

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains; the terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention; the term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0042] In real-world roads, there are stable and clear topological and geometric constraints between the curb and lane lines (e.g., they are often approximately parallel on straight roads and converge at ramp entrances). Vehicle lateral positioning, not limited to lane line information, suffers from rich correlations with the curb, which are not directly used to guide, verify, or supplement curb recognition results. This results in the system lacking the ability to make reasonable inferences using known reliable information (clear lane lines) when curb observation information is ambiguous or missing. Furthermore, at intersections, channelization islands, or construction zones, the curb's direction may undergo legitimate, drastic changes or even interruptions. Algorithms relying solely on historical state smoothing struggle to distinguish between such "legitimate scene structure changes" and "abnormal jumps caused by perceived noise," easily leading to tracking loss or smoothing results that violate common sense about road topology.

[0043] Therefore, existing roadside recognition technologies suffer from problems such as perception interruption due to occlusion and inability to make reasonable inferences due to a lack of scene knowledge. These issues make it difficult to meet the stringent requirements of autonomous vehicles for high robustness and continuity of perception systems in highly complex and dynamically changing urban environments. There is an urgent need for a novel roadside recognition method that can fully utilize the inherent constraints between multiple elements in a road scene, thereby maintaining reasonable output even under conditions of missing observations.

[0044] Using lane line detection results as an auxiliary reference for curb recognition, the topological relationship pattern between curb and lane line is identified in real time, and constraint rules are constructed based on this relationship pattern. When curb observation is unreliable, reasonable curb state is inferred using reliable lane line information, thereby improving the continuity and scene adaptability of curb recognition.

[0045] Example 1: Figure 1 This is a schematic flowchart of an image-based curb recognition method according to an embodiment of the present invention.

[0046] like Figure 1 As shown, an image-based road edge recognition method includes the following steps:

[0047] S101: Obtain the road scene image of the current frame, input it into the pre-trained road semantic segmentation model, output the road semantic label map, and use the preset road observation mechanism to obtain the initial roadside observation point sequence of the current frame and its corresponding observation confidence evaluation value, as well as the set of lane line geometric parameters.

[0048] Specifically, RGB images of the road scene ahead are acquired using a monocular or binocular camera mounted on the autonomous vehicle at a fixed frequency (e.g., 10Hz or 30Hz). It should be noted that the road scene images have already undergone preprocessing, including distortion correction, which will not be elaborated upon in this invention.

[0049] In some embodiments, the pixels in the road semantic labeling map are at least classified into drivable road areas, non-road areas, and lane line labels. Specifically, the road semantic labeling map... Each pixel value For each category ID, the preset key category IDs must include at least: drivable road surface area. Non-road areas (Including non-road surface areas such as curbs, sidewalks, green belts, and obstacle bases), lane lines .

[0050] For example, the pre-trained road semantic segmentation model is trained on a large number of historical road scene images and road scene images labeled with semantic categories (i.e., semantic label images). As an example, the model adopts an encoder-decoder architecture (such as DeepLabV3+, HRNet, etc.). The encoder is used to extract multi-level features, the decoder is used to restore spatial resolution and perform pixel-level classification, and the last layer of the model is a softmax classification layer, which outputs the probability of each pixel belonging to a preset category. By selecting the category with the highest probability for each pixel, a semantic label map with the same resolution as the input image is obtained. For the specific application process of the semantic segmentation model, please refer to relevant existing technologies, which will not be elaborated in this invention.

[0051] In some embodiments, the preset road observation mechanism specifically includes:

[0052] S201 involves performing boundary detection processing on the semantic label map to extract the boundary pixels between the drivable area and the non-road area. After coordinate transformation and denoising, the initial roadside observation point sequence of the current frame is formed, and the observation confidence evaluation value corresponding to the sequence is calculated.

[0053] Specifically, extracting the boundary pixels between the drivable area and the non-road area includes: traversing the semantic label map, for any pixel coordinates... ,like (For drivable road areas), check if there are any conditions that satisfy the condition within its 8 neighboring areas. pixels If it exists, then set the current pixel. These are marked as road surface-to-non-road surface boundary pixels. The essence of boundary pixel recognition is to extract the contour of the road surface region.

[0054] Specifically, the initial roadside observation point sequence for the current frame, after coordinate transformation and denoising, includes:

[0055] pixel coordinates of all detected boundary pixels By mapping the image pixel coordinates to two-dimensional coordinates in the vehicle coordinate system, a set of candidate roadside points is obtained. , The vertical distance is... Horizontal distance;

[0056] Based on the candidate roadside point set, a distance-based clustering algorithm (DBSCAN) is used to remove outliers that are significantly far from the main point group (this invention will not elaborate on this; please refer to relevant technical descriptions of clustering algorithms for details). This yields a valid point set, which is then sorted according to the x-coordinates of the points. (In the direction of vehicle travel) The observation points are sorted from smallest to largest to form an ordered sequence of initial roadside observation points. ,in N is the number of points in the initial roadside observation point sequence.

[0057] It should be noted that the coordinate transformation specifically involves using the camera intrinsic parameter matrix K and pre-calibrated extrinsic parameters (camera mounting height relative to the vehicle body, pitch angle, etc.) to transform the image coordinates to the vehicle coordinate system (typically with the rear axle center of the vehicle as the origin, the X-axis pointing forward, the Y-axis pointing to the left, and the Z-axis pointing upward). Alternatively, one can refer to existing technologies describing the application of IPM; this invention will not elaborate further. For example, IPM requires a pre-calibrated camera intrinsic parameter matrix K, extrinsic parameters (camera rotation matrix R and translation vector T relative to the vehicle body), and an assumed ground level equation. The transformation formula is as follows: Where d is a preset scale factor, on the assumed ground plane When =0, the solution can be obtained, and the final result is... .

[0058] The transformed point set may contain noise (due to segmentation errors or edge distortion from perspective transformation), so denoising is required.

[0059] Specifically, the calculation of the observation confidence assessment value corresponding to the sequence includes:

[0060] Based on the initial roadside observation point sequence, the boundary integrity, point sequence distribution density, and boundary contrast are extracted, and the observation confidence evaluation value is obtained through comprehensive evaluation.

[0061] For example, , This is the observation confidence assessment value. To calculate the distribution density of the point sequence, calculate the average interval of the point sequence on the horizontal axis. It is used to evaluate the average density of a point sequence in the X direction; the higher the density, the higher the confidence level. Boundary integrity is used to assess the coverage of the initial roadside point set within the effective longitudinal distance range. First, determine the effective longitudinal range of the curb detection. and the theoretical maximum number of sampling points within this range , The effective longitudinal range is determined by the length of the effective longitudinal range and a preset, reasonable longitudinal sampling interval. Decision, that is , The boundary integrity can be set according to the sensor accuracy and application requirements (e.g., 0.1 meters). The boundary integrity is then defined as the sum of the actual number of valid points N extracted (the number of points in the initial roadside observation point sequence) and the theoretical maximum number of sampling points. The ratio of and saturation treatment: This ratio directly reflects the longitudinal coverage integrity of curb detection. The higher the ratio, the more complete the coverage and the higher the confidence level. To set the boundary contrast, set it as the output probability map of the road semantic segmentation model. At all boundary pixels The average value is the difference in probability between the boundary pixels belonging to the drivable road surface pixels and the non-road surface pixels at all identified boundary pixels during the boundary extraction process. The larger the average probability difference, the more certain the model is in judging the boundary and the higher the confidence level. , , These correspond to weighting coefficients, which are set based on expert experience and practical scenario adjustments, with the sum of the weighting coefficients being 1. It should be noted that normalization processing is required before performing the weighted summation to unify the units of measurement; this will not be elaborated upon in this invention.

[0062] S202, cluster and parametric curve fitting are performed on the lane line category pixels in the semantic label map to obtain the set of lane line geometric parameters for the current frame.

[0063] Specifically, step S202 includes:

[0064] A1. From the road semantic label map Extract all pixels belonging to the lane line category and transform them to the vehicle coordinate system to obtain the lane line pixel point cloud. Then, use a clustering algorithm based on Euclidean distance (such as K-Means or hierarchical clustering based on distance thresholds) to divide the point cloud into K clusters. Each cluster corresponds to an independent lane line.

[0065] It should be noted that since the image may contain multiple lane lines, these pixels need to be clustered so that the pixels of each physical lane line belong to an independent cluster. Clustering can be done based on the vertical position (X coordinate) and horizontal position (Y coordinate) of the pixels in the vehicle coordinate system, using algorithms such as distance-based clustering.

[0066] A2. For each cluster Perform curve fitting using a predefined parametric curve model (e.g., a cubic polynomial). The algorithm is fitted to the lane, and the fitting process requires a robust method (such as the Random Sample Consensus Algorithm RANSAC) to resist the interference of segmentation noise. After successful fitting, the geometric parameter vector describing the lane line is obtained, including polynomial coefficients. The combined parameter vector of the k-th lane line is expressed as... .in These are the image coordinates.

[0067] RANSAC, through random sampling, model solving, and iterative interior point calculation, ultimately finds the model parameters with the largest number of interior points. The threshold for determining the interior point is usually set to 0.05-0.15 meters (lateral distance deviation in the vehicle coordinate system, which is pre-calibrated according to the actual scenario). For specific details, please refer to the relevant existing technical principles and applications. This invention will not elaborate on this.

[0068] A3. Calculate the lane confidence score for each fitted lane line. The lane confidence score is based on a comprehensive assessment of the proportion of interior points used to fit the curve, the fitting residuals, and the number of original pixels within the cluster.

[0069] For example, the lane confidence score is calculated according to the following formula:

[0070]

[0071] in, The number of points within RANSAC. This represents the total number of points within the cluster. denoted as the root mean square error of the inlier points relative to the fitted curve. The maximum acceptable error limit is preset based on expert experience and real-world scenario data.

[0072] A4. Construct a set of lane geometry parameters from all lane lines and their lane confidence scores. , represented as .

[0073] S102, determine the reference lane line from the set of lane line geometric parameters, and identify the spatial constraint pattern between the initial roadside observation point sequence and the reference lane line. The spatial constraint pattern includes at least the parallel constraint pattern and the convergence / separation constraint pattern.

[0074] Specifically, step S102 includes:

[0075] At least one lane line with the highest spatial correlation to the initial roadside observation point sequence is selected as the reference lane line. Based on the relative position distribution between the initial roadside observation point sequence and the reference lane line, the roadside-lane line spatial constraint pattern of the current frame is identified. The spatial constraint pattern includes at least the parallel constraint pattern and the convergence / separation constraint pattern. The spatial constraint strength index of the current frame is calculated.

[0076] In some embodiments, the method for selecting the reference lane line is as follows:

[0077] For each lane line in the set of lane line parameters Calculate the initial roadside observation point sequence The average lateral distance from all points to the lane curve , x-axis The ordinate corresponding to the curve function of the lane line is selected such that... The smallest lane line as a reference lane line . Lane lines The curve function.

[0078] It should be noted that when selecting a reference lane line, the condition can also be set to: make Minimum and its lane confidence Higher than the preset lane confidence threshold (Pre-set lane lines based on expert experience and historical data) If multiple lane lines are close together, the lane with the highest confidence level is selected.

[0079] In some embodiments, the roadside-lane spatial constraint pattern of the current frame is identified based on the relative positional distribution between the initial roadside observation point sequence and the reference lane line, specifically including:

[0080] S301, based on reference lane lines Calculate the initial roadside observation point sequence Lateral offset sequence relative to the reference lane line.

[0081] Along the reference lane line Within its preset effective longitudinal range Within, at preset fixed intervals Sample M longitudinal positions For each In the initial roadside observation point sequence Find its x-coordinate in the middle With the current sampling point The curb point with the smallest absolute difference is denoted as . and calculate At this place ( Horizontal coordinate of (location) , thus obtaining the horizontal offset sequence , For the corresponding curb point The horizontal coordinate, For reference, the lane lines are in the longitudinal position. The horizontal coordinates of each point in the horizontal offset sequence Indicates: at a specific longitudinal position in front of the vehicle. At this point, the lateral offset of the curb point relative to the reference lane line.

[0082] Among them, the effective longitudinal range Defined as: on the X-axis of the vehicle coordinate system, define an interval of interest. Set the nearest point to the vehicle bumper location (e.g., 0 meters or a small positive value to avoid the vehicle body itself obstructing the view). The furthest point is set, determined by the camera's effective sensing distance, segmentation model accuracy, or actual planning requirements (e.g., 30 meters). The effective longitudinal range is used to define the spatial scope of the analysis. A fixed interval is preset. In order to be in The fixed step size for sampling along the X-axis within the range (set based on expert experience with the road, for example, 0.2 meters), where M is the total number of sampling points.

[0083] S302, based on the lateral offset sequence, determines the spatial constraint mode.

[0084] Specifically, it includes:

[0085] B1. Calculate the variance of the lateral offset sequence D. ,in The mean of the horizontally offset sequence, if ( The preset variance threshold is set based on expert experience and real-world scenarios; for example, it could be set to 0.02. ),and (A preset reasonable distance range between the curb and the lane line is set based on the actual scenario and experts, such as 0.3m to 4m), then the current frame's curb-lane line is determined to be in parallel mode. .

[0086] B2. For all data pairs (i.e., the offset sequence D with respect to the vertical coordinate) performs first-order linear regression. The slope s, intercept b, and coefficient of determination for the fit are obtained. (This is a core indicator in statistics used to measure the goodness of fit of a linear regression model, quantifying how closely data points fit the fitted line.) ( The preset trend intensity threshold is used, for example, set to 0.03, which means a change of 3 centimeters for every 1 meter of forward movement. ( The preset trend significance threshold is an empirical threshold, for example, set to 0.6. Setting a higher threshold means stricter requirements for the quality of linear relationships, which can reduce the possibility of misjudging nonlinear changes or noise as convergence / separation patterns. Then, the slope sign is used to determine: if s is less than 0, it is a convergence pattern. (The curb gradually approaches the lane line); if s is greater than 0, it is in separation mode. (The curb gradually moves away from the lane lines).

[0087] It should be noted that if none of the above conditions are met, it is judged as an unknown mode, denoted as .

[0088] It should be noted that the determination coefficient This measure indicates the degree to which distance d changes linearly with longitudinal position X; a value closer to 1 indicates a more significant linear relationship. In real-world road scenarios, the distance between the curb and lane lines is sometimes not a perfect straight line (e.g., in S-curves). By introducing... A threshold can prevent erroneous linear pattern judgments on these complex curve segments, thus more reliably classifying them as "unknown patterns." The calculation of the determination coefficient can be referenced from existing technologies describing the application of linear regression fitting; this invention will not elaborate on this. Exemplarily, , , The sum of squares (S) represents the predicted values ​​from the linear regression model. The numerator is the sum of squared residuals, which represents the sum of squared differences between the observed and predicted values, indicating the portion of variation that the model failed to explain. The denominator is the total sum of squares, representing the original data. Relative to its mean The overall degree of dispersion. In actual programming, the determination coefficient value is usually obtained by directly calling the linear regression function.

[0089] It should be noted that the various judgment thresholds involved in this embodiment (the aforementioned preset variance thresholds) Trend strength threshold Significant trend threshold The threshold parameters are all empirical values ​​determined through statistical analysis and engineering experience debugging based on a large amount of offline experimental data from real road scenarios. In practical applications, they can be adaptively calibrated and adjusted according to the accuracy of the vehicle-mounted sensor, vehicle dynamics characteristics, and target application scenarios (such as urban roads and parks) to balance the sensitivity and robustness of recognition. For example, the setting is based on the following: In the vehicle coordinate system, for real roadside samples containing "parallel" relationships, the variance distribution of their lateral offset sequence is calculated, and the 90th percentile of this distribution is taken as a reference value. After verification by actual road tests, it is determined to be 0.02. ; The setting is based on the fact that the statistical mean of the rate of change of the distance between the road edge and the lane line in convergence or separation scenarios is about 0.03, which is used as the threshold to distinguish between trend and noise. The setting is based on the fact that the coefficient of determination for linear regression fitting is usually higher than 0.6 in real convergence / separation scenarios, so this value is taken as the threshold for determining the significance of linear trends.

[0090] S303, Calculate the spatial constraint strength index of the current frame based on the identified spatial constraint pattern:

[0091] For parallel patterns, define an intensity index. , The smaller the variance, the closer Q is to 1, indicating a stronger and more stable parallel relationship.

[0092] For convergence / separation patterns, define an intensity index. , The normalization factor is defined (e.g., 0.1). The larger the absolute value of the slope, the more obvious the trend, and the larger Q is.

[0093] For unknown patterns, the intensity index Q=0 is defined.

[0094] Thus, the complex spatial relationship between the curb and lane lines is extracted into a clear semantic category and a continuous intensity value, realizing a quantitative expression of high-level scene understanding.

[0095] S103, based on the time series feature vector constructed from the spatial constraint patterns of multiple consecutive frames within a preset window (set to a relatively short window), evaluate the comprehensive confidence of the spatial constraint relationship of the current frame.

[0096] Specifically, step S103 includes:

[0097] A time-series feature vector is constructed based on the spatial constraint pattern and spatial constraint strength index of multiple consecutive frames within a preset window. This vector is then input into a pre-trained temporal relationship evaluation model. The output represents the temporal stability score of the spatial constraint pattern and the anomaly confidence score, indicating whether the relationship features of the current frame are abnormal. In combination with the spatial constraint strength index, a comprehensive confidence score is calculated to characterize the credibility of the spatial constraint relationship of the current frame.

[0098] In some embodiments, the time series feature vector is set as follows:

[0099] C1. For each frame t within the preset window (the historical window before the current frame), construct the frame feature vector. Including spatial constraint modes One-hot coding and spatial constraint strength index Observation confidence assessment value , Correlation geometric statistics (including at least one or more of the mean and standard deviation of the horizontally offset series (if it is a parallel or convergent / separate pattern)).

[0100] C2. Construct a temporal feature matrix from the frame feature vectors of L consecutive frames within the preset window. L is the number of frames, and H is the dimension of the frame feature vector.

[0101] In some embodiments, the pre-trained temporal relation evaluation model can be configured as a lightweight temporal model (such as a one-dimensional convolutional network or a gated recurrent unit GRU) that outputs two key scalars:

[0102] Stability rating : Indicates prediction of the current pattern based on the historical pattern sequence. The probability of the current spatial constraint pattern remaining unchanged within a future time window (short term 3-5 frames) is used as the supervision signal during training. This supervision signal is the future... An indicator of whether the intra-frame mode has changed (1 if it remains unchanged, 0 otherwise).

[0103] Anomaly confidence : Represents the frame feature vector of the current frame The degree of anomalousness relative to the “normal” historical pattern sequence learned by the model indicates whether the relational features of the current frame are inconsistent with the historical pattern; during training, the supervision signal of this output is whether it is marked as anomalous (e.g., by offline analysis, frames with abrupt pattern changes and no reasonable geometric interpretation are marked as anomalous).

[0104] It should be noted that the training data for the temporal relationship evaluation model comes from a large number of real driving scenarios. By labeling the time-series feature vectors of historical frames, the pattern label and anomaly label of each frame are obtained, and the model is trained in a supervised learning manner.

[0105] For example, the training method for the time series relationship evaluation model can be:

[0106] D1. Collect a large number of historical road scene images of autonomous vehicles. For each frame of image, generate its corresponding time series feature vector and label it.

[0107] For example, on an autonomous vehicle (such as a sweeper), the continuous video stream collected by the onboard camera is recorded simultaneously when the vehicle is driving in different working scenarios (including urban auxiliary roads, internal roads of parks, park paths, and road sections including intersections / ramp). The video stream also includes the vehicle's basic positioning and attitude information (such as small amplitude displacement data from wheel speedometers and IMUs). The data collection process needs to cover various lighting conditions (daytime, dusk, shadow), weather (sunny, cloudy), and different degrees of roadside occlusion (temporarily blocked by pedestrians, vehicles, and vegetation).

[0108] The labeling includes stability score labels and anomaly confidence labels:

[0109] Generation of stability rating labels:

[0110] Definition: The true value of the stability score should represent the probability that the current spatial constraint pattern will remain unchanged in the short term.

[0111] Labeling rule: For the corresponding frame t, examine the spatial constraint pattern sequence of W frames within its subsequent time window. ; Calculate the pattern in frame W and the current frame Same frame rate Set a ratio threshold (e.g., 0.8); if If the current pattern is stable, it is considered stable and assigned a stability label of 1; otherwise, it is considered unstable and assigned a label of 0. This corresponds to scenarios where the topology of roads usually does not change drastically within a very short time window.

[0112] Generation of anomaly confidence labels:

[0113] Definition: The true value of anomaly confidence should indicate whether the relational features of the current frame deviate from the "normal" pattern represented by the historical context;

[0114] Labeling rules: An initial version of the labels can be automatically generated using an unsupervised method that reconstructs errors based on historical context, and then supplemented with manual verification and correction.

[0115] For a time series feature vector consisting of L consecutive frame feature vectors An autoencoder model with an encoder-decoder structure is used to learn to reconstruct the sequence. This model is trained on a large amount of normal data, with the goal of learning to compress and reconstruct common, stationary temporal patterns. On the data used to train the main model, each segment of the sequence is processed with the trained autoencoder, and the reconstruction error of the feature vector of the current frame is calculated. (Mean Squared Error) Calculate the distribution of reconstruction error across the entire dataset, and set a percentile threshold (e.g., 95th percentile). If the value exceeds the percentile threshold, the frame is initially marked as abnormal and assigned a value of 1; otherwise, it is considered normal and assigned a value of 0. The automatically marked abnormal samples are manually sampled to determine whether they actually correspond to real abnormal situations such as severe roadside obstruction, lane line misdetection, or severe vehicle bumps. Samples that are mislabeled (actually normal but with high error) are corrected. At the same time, some high-risk scenarios (such as intersection transition areas) that are automatically marked as "normal" are manually checked to ensure that the labels are accurate.

[0116] It should be noted that labeling can usually be done manually by combining actual road scenarios and expert experience, and quantified to 0-1. The above is just an example, and this invention will not elaborate on it.

[0117] D2. Using the feature vectors of all labeled event sequences as the training dataset, train the pre-selected neural network structure, continuously optimize the model parameters, and obtain the final time-series relationship evaluation model.

[0118] For example, the pre-selected neural network structure can be a lightweight temporal convolutional network. The prepared training dataset is divided into training, validation, and test sets in chronological order (note that the sequence continuity should be maintained to avoid data leakage caused by random shuffling). The Adam optimizer is used to perform iterative training on the training set, and the weighted loss function (loss for stability prediction and loss for anomaly detection, respectively assigned to task weights to balance the importance of the two tasks, usually set through grid search or based on the performance of the validation set, both using binary cross-entropy loss) is monitored on the validation set. An early stopping strategy is adopted, and training is stopped when the validation set loss no longer decreases over several consecutive periods. The model weights with the best overall performance on the validation set are selected.

[0119] For example, a two-layer gated recurrent unit (GRU) is used as the temporal relationship evaluation model. The input is the temporal feature matrix. After processing by the GRU, the output of the last time step is connected to two independent fully connected layers to output the stability score and the anomaly confidence, respectively. During training, the Adam optimizer is used with an initial learning rate of 0.001, a batch size of 64, and 50 epochs of iterative training. An early stopping strategy is adopted (the model stops if the validation set loss does not decrease for 10 consecutive epochs). The loss function is the binary cross-entropy loss. The weights of the two tasks are both set to 1.0, and the weights are equal to balance the importance of the two tasks. Finally, the model weight with the smallest weighted loss on the validation set is selected as the final temporal relationship evaluation model.

[0120] In some embodiments, a comprehensive confidence score is calculated to characterize the credibility of the spatial constraint relationship in the current frame, specifically including:

[0121]

[0122] in, To assess the overall confidence level, For stability rating, For abnormal confidence levels, As a spatial constraint strength index, For adjustable weighting coefficients, satisfying For example, (0.4, 0.3, 0.3). Give high weight to timing stability. Convert the abnormality level to the normality level. Then consider the instantaneous relationship strength of the current frame.

[0123] S104. Based on the comparison between the observed confidence level and the overall confidence level, the final road edge curve is generated using a preset routing strategy. The discretized set of coordinate points.

[0124] In some embodiments, the preset routing policy is based on a preset threshold interval. , Set a threshold for the first one. Regarding the second threshold setting, it should be noted that both the first and second thresholds are empirical values ​​determined through statistical analysis and engineering debugging based on extensive offline experimental data from real-world road scenarios. These thresholds can be adaptively calibrated and adjusted according to the characteristics of the vehicle-mounted sensors and the application scenario. Preferably, Specifically, this includes:

[0125] S401, if the observation confidence assessment value is higher than the first set threshold, then the observation-led route is adopted: robust curve fitting is performed on the initial roadside observation point sequence to directly generate the final roadside curve.

[0126] Specifically, when When the observation quality is high, an observation-driven route is adopted: robust curve fitting is performed on the initial roadside observation point sequence. For example, a quadratic or cubic polynomial curve y=g(x) is fitted using the RANSAC algorithm. Since the point set may contain noise, RANSAC can robustly estimate the curve parameters through random sampling and consensus set filtering. The fitted curve is denoted as . .

[0127] Spatial constraint patterns are only used as an aid; for example, during RANSAC random sampling, priority is given to selecting lanes whose distance from the reference lane line matches the current pattern. The points are used as initial samples to improve fitting efficiency and rationality.

[0128] S402, if the observation confidence assessment value is lower than the second set threshold and the comprehensive confidence is higher than the first set threshold, then structural inference routing is adopted: based on the spatial constraint mode of the current frame and the geometric parameters of the reference lane line, an inferred roadside curve is generated through predefined geometric mapping rules as the final roadside curve.

[0129] Specifically, when At that time, if the observation quality is considered poor but the spatial constraint relationship is reliable, structural inference routing is adopted: based on the spatial constraint pattern of the current frame. Curve function of reference lane line The inferred road edge curve is generated using set mapping rules (predefined geometric mapping rules):

[0130] like The average lateral offset of the roadside-lane line is calculated from historical reliable frames using translational inference. (For example, taking the average offset of the most recent reliable frame), the inferred roadside curve is: Soon constant term offset The criteria for determining historically reliable frames are as follows: within a preset historical time window (e.g., the most recent 20 frames), frames with observation confidence assessment values ​​higher than a first set threshold are selected as reliable frames. If at least one reliable frame exists, the average of the mean values ​​of the lateral offset sequences among these reliable frames is calculated as the reference offset. If no reliable frame is available, the offset estimated in the current frame (i.e., the mean of the lateral offset sequence in the current frame) is used as the reference offset, or a preset typical curb-lane distance (e.g., 1.5 meters, which can be calibrated according to vehicle type and work scenario) is used.

[0131] like It employs trend extrapolation inference, using a offset linear model fitted with current frame and historical frame data. Then, for any X, the inferred Y-coordinate of the roadside point is: , To reference the Y value of the lane line at point X, use a set of Point refit yields the inferred roadside curve ;

[0132] If the reference lane line does not exist or the pattern is unknown, structural reasoning cannot be performed. You can fall back to using the historical road edge curve (such as the result of the previous frame) or output an invalid result.

[0133] S403, if both the observation confidence assessment value and the comprehensive confidence value are in the intermediate transition range, then a hybrid fusion route is adopted: the fitted curve generated by the observation-driven route and the inferred path curve generated by the structural inference route are weighted and fused to generate the final path curve.

[0134] Specifically, when Hybrid routing is used:

[0135] Fitted curves obtained by observing the dominant route The inference curve is obtained through structural reasoning routing. Perform weighted fusion:

[0136]

[0137] in The preset balance factor is used to adjust the weight ratio of the two sources. Its value can be determined by debugging according to the actual scenario. In this embodiment, it is set to 1.2, which means that under the same confidence level, the inference result is given a slightly higher weight. For the fitted curve The coefficient vector, For inference curves The coefficient vector of the curve, and the corresponding addition and multiplication operations are both element-wise operations performed on the corresponding elements of the coefficient vector (corresponding polynomial coefficients) of the curve (see the following example). This is the final curb curve.

[0138] As an example of the parametric form of a curve, the curb curve is represented using a cubic polynomial model, meaning the curve equation can be expressed as: ,in, Let be the coefficient vector of the curve. Based on this, let the fitted curve obtained from the observation-dominant route be... The coefficient vector is The inference curve obtained from the structural inference route The coefficient vector is The final roadside curve coefficient vector The result is obtained by weighted averaging of the two coefficient vectors mentioned above.

[0139] It should be noted that if none of the above conditions are met, such as low observation quality and low relationship reliability, the road edge curve of the previous frame can be used or a system alarm can be triggered.

[0140] In some embodiments, generating the discretized coordinate point set of the final road edge curve specifically includes:

[0141] The final curb curve (such as) Discretization is performed to generate a series of discrete point coordinates for use by the planning and control modules. A discretization method could be as follows: In the vehicle coordinate system, along the X-axis (the vehicle's forward direction), within a preset range of distances of interest (e.g., 0 to 30 meters), samples are taken at fixed intervals (0.5 meters) that meet the control frequency requirements, and the coordinates of each sample point are calculated. Corresponding horizontal coordinate This generates an ordered set of discrete points. O is the number of points in the discrete point set;

[0142] At the same time, the point set The data is sent to the vehicle's planning and control module via a vehicle communication bus (such as CAN or Ethernet). This module uses these discrete points to generate a smooth driving trajectory that fits the road edge or to calculate lateral deviations. Confidence information (observation confidence, relational comprehensive confidence) and spatial constraint patterns of the current frame can be additionally output for downstream modules to reference.

[0143] The technical solutions described in the embodiments of this application have at least the following technical effects or advantages:

[0144] Through a multi-class semantic segmentation model, synchronous and dense identification of road surface, non-road surface areas, and lane lines in road scenes is achieved. This eliminates the data asynchrony and calibration error problems caused by traditional multi-model parallelism, ensuring the consistency of roadside and lane line information in time, space, and geometry from the source. The output initial roadside observation point sequence and lane line geometric parameter set are not only direct inputs for subsequent analysis, but their inherent consistency is also the fundamental guarantee for the subsequent construction of highly reliable spatial constraint relationships. The calculated observation confidence is the first to quantify the original quality of single-frame roadside perception, providing a key "observation-side" confidence basis for subsequent decision-making.

[0145] By dynamically associating the most relevant lane lines as a reference benchmark and systematically quantifying the lateral offset of the road edge relative to this benchmark, the question of "where is the road edge?" is transformed into the question of "what spatial pattern do the road edge and lane lines form?", resulting in clear spatial constraint patterns (such as parallel and converging) and continuous intensity indicators.

[0146] By modeling and analyzing the spatial constraint patterns of historical continuous frames, a pre-trained temporal model is used to intelligently evaluate the stability of the current relationship pattern and whether there are any instantaneous anomalies. This upgrades the static spatial relationship judgment of a single frame to a dynamic credibility assessment based on historical context. The output stability score and anomaly confidence score are combined to generate an authoritative index representing the credibility of the "relationship side," which is a comprehensive confidence score. This enables the distinction between "long-term stable road structure relationships" and "pseudo-relationships caused by instantaneous occlusion or noise."

[0147] Based on the observation confidence and the comprehensive confidence, multimodal condition judgment is performed, and the system adaptively selects and executes three routing strategies. When the observation quality is low but the scene structure relationship is clear and reliable, it can actively "infer" a virtual curb that conforms to the common sense of road geometry based on the identified topological pattern and high-precision lane line information. This changes the passive situation of traditional methods that can only extrapolate or lose data under severe occlusion, and realizes "uninterrupted" perception. Whether it is observation-driven accurate fitting, intelligent completion of relational reasoning, or smooth fusion of the two, it can always output the most reasonable and reliable curb curve in the current situation, which significantly improves the system's survivability and output continuity under extreme conditions and is a direct manifestation of the robustness of the solution.

[0148] In summary, by constructing a closed-loop framework from unified perception to intelligent decision-making, the continuity, robustness, and rationality of roadside boundary perception in complex and dynamic road scenarios of autonomous vehicles are significantly improved. First, a unified semantic segmentation model is used to jointly analyze road images, synchronously and consistently extracting the initial observation points of the roadside and the geometric parameters of the lane lines, laying a spatiotemporally aligned data foundation for subsequent analysis. Then, by quantifying and analyzing the spatial geometric relationship between the roadside point set and the reference lane lines, the complex scene structure is transformed into explicit topological constraint patterns (such as parallelism and convergence) and their strength indices. Based on this, a temporal prediction model is introduced to intelligently evaluate the historical evolution of the above constraint relationships, outputting their stability and anomaly degree, comprehensively forming an authoritative judgment on the credibility of the current structural relationships. Finally, the system... Based on the real-time calculated confidence scores of roadside observations and the comprehensive confidence scores of structural relationships, the system adaptively switches and executes three routing strategies: "observation-driven fitting," "structural relationship inference," and "weighted fusion." This ensures perception accuracy when roadside visibility is good, and when severe occlusion causes observation failure, it can rely on a deep understanding of the road topology to generate physically reasonable virtual roadsides based on highly reliable lane line information. This effectively solves the problems of perception interruption and misjudgment in extreme situations such as occlusion in traditional methods, providing stable and reliable perception input for safe and smooth roadside operation.

[0149] Furthermore, embodiments of the present invention also provide an image-based curb recognition system.

[0150] Figure 2 This is a structural block diagram of an image-based curb recognition system according to an embodiment of the present invention.

[0151] like Figure 2 As shown, an image-based curb recognition system includes: an acquisition module, a recognition module, an evaluation module, and a curb generation module;

[0152] The acquisition module is used to acquire the road scene image of the current frame, input it into the pre-trained road semantic segmentation model, output the road semantic label map, and use the preset road observation mechanism to obtain the initial roadside observation point sequence of the current frame and its corresponding observation confidence evaluation value, as well as the set of lane line geometric parameters.

[0153] The identification module is used to determine the reference lane line from the set of lane line geometric parameters and identify the spatial constraint pattern between the initial roadside observation point sequence and the reference lane line. The spatial constraint pattern includes at least the parallel constraint pattern and the convergence / separation constraint pattern.

[0154] The evaluation module is used to evaluate the comprehensive confidence of the spatial constraint relationship of the current frame based on the time series feature vector constructed from the spatial constraint pattern of multiple consecutive frames within a preset window.

[0155] The curb generation module is used to generate a set of discretized coordinate points of the final curb curve based on the comparison results of the observation confidence and the comprehensive confidence, using a preset routing strategy.

[0156] It should be noted that other specific implementation details of the embodiments of the present invention can refer to the above-described image-based curb recognition method.

[0157] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image-based road edge recognition method, characterized in that, include: S101: Obtain the road scene image of the current frame, input it into the pre-trained road semantic segmentation model, output the road semantic label map, and use the preset road observation mechanism to obtain the initial roadside observation point sequence of the current frame and its corresponding observation confidence evaluation value, as well as the set of lane line geometric parameters. S102, determine the reference lane line from the set of lane line geometric parameters, and identify the spatial constraint pattern between the initial roadside observation point sequence and the reference lane line. The spatial constraint pattern includes at least the parallel constraint pattern and the convergence / separation constraint pattern. S103, Evaluate the comprehensive confidence of the spatial constraint relationship of the current frame based on the time series feature vector constructed from the spatial constraint pattern of multiple consecutive frames within a preset window; S104. Based on the comparison between the observed confidence level and the overall confidence level, a set of discretized coordinate points for the final roadside curve is generated using a preset routing strategy.

2. The image-based road edge recognition method as described in claim 1, characterized in that, The pixels in the road semantic label map are classified into at least the drivable road surface area, the non-road surface area, and the lane line label.

3. The image-based road edge recognition method as described in claim 2, characterized in that, The preset road observation mechanism specifically includes: S201, boundary detection processing is performed on the semantic tag map to extract the boundary pixels between the drivable area and the non-road area. After coordinate transformation and denoising, the initial roadside observation point sequence of the current frame is formed, and the observation confidence evaluation value corresponding to the sequence is calculated. The observation confidence evaluation value is obtained by extracting boundary integrity, point sequence distribution density, and boundary contrast based on the initial roadside observation point sequence. S202, cluster and parametric curve fitting are performed on the lane line category pixels in the semantic label map to obtain the set of lane line geometric parameters for the current frame.

4. The image-based curb recognition method as described in claim 3, characterized in that, The set of lane line geometry parameters includes the curve parameters of at least one lane line and its corresponding lane confidence level.

5. The image-based road edge recognition method as described in claim 4, characterized in that, The method for selecting the reference lane line is as follows: For each lane line in the set of lane line geometry parameters Calculate the average lateral distance from all points in the initial roadside observation point sequence to the lane line curve. Select to make The smallest lane line as a reference lane line .

6. The image-based road edge recognition method as described in claim 4, characterized in that, The identification of the spatial constraint pattern between the initial roadside observation point sequence and the reference lane line specifically includes: S301, based on reference lane lines Calculate the initial roadside observation point sequence Lateral offset sequence D relative to the reference lane line: along the reference lane line Within its preset effective longitudinal range Within, at preset fixed intervals Sample M longitudinal positions For each In the initial roadside observation point sequence Find its x-coordinate in the middle With the current sampling point The curb point with the smallest absolute difference is denoted as . and calculate The horizontal coordinate at this location , thus obtaining the horizontal offset sequence , For the corresponding curb point The horizontal coordinate, For reference, the lane lines are in the longitudinal position. The horizontal coordinate of the location; S302, Based on the lateral offset sequence, determine the spatial constraint mode; S303, calculate the spatial constraint strength index of the current frame based on the identified spatial constraint pattern.

7. The image-based road edge recognition method as described in claim 6, characterized in that, S302 specifically includes: B1. Calculate the variance of the lateral offset sequence D. ,in The mean of the horizontally offset sequence, if ,and , The preset variance threshold, If the preset reasonable distance range between the curb and lane line is met, then the current frame's curb-lane line is determined to be in parallel mode. ; B2. For all data pairs Perform a first-order linear regression to obtain the slope s, intercept b, and coefficient of determination for the fit. ,like and , The preset trend strength threshold, If a predefined threshold for trend significance is used, the slope sign is used to determine the convergence pattern: if s is less than 0, then it is a convergence pattern. If s is greater than 0, then it is in separation mode. .

8. The image-based curb recognition method as described in claim 7, characterized in that, S103 specifically includes: A time-series feature vector is constructed based on the spatial constraint pattern and spatial constraint strength index of multiple consecutive frames within a preset window. This vector is then input into a pre-trained temporal relationship evaluation model. The output represents the temporal stability score of the spatial constraint pattern and the anomaly confidence score, indicating whether the relationship features of the current frame are abnormal. In combination with the spatial constraint strength index, a comprehensive confidence score is calculated to characterize the credibility of the spatial constraint relationship of the current frame.

9. The image-based curb recognition method as described in claim 8, characterized in that, The time series feature vector is set as follows: C1. For each frame t within the preset window, construct the frame feature vector. Including spatial constraint modes One-hot coding and spatial constraint strength index Observation confidence assessment value Correlation geometric statistics; correlation geometric statistics include at least one or more of the mean and standard deviation of the horizontally offset series; C2. Construct a temporal feature matrix from the frame feature vectors of L consecutive frames within the preset window. L is the number of frames, and H is the dimension of the frame feature vector.

10. An image-based curb recognition system, applied to an image-based curb recognition method as described in any one of claims 1 to 9, characterized in that, The system includes: an acquisition module, an identification module, an evaluation module, and a curb generation module; The acquisition module is used to acquire the road scene image of the current frame, input it into the pre-trained road semantic segmentation model, output the road semantic label map, and use the preset road observation mechanism to obtain the initial roadside observation point sequence of the current frame and its corresponding observation confidence evaluation value, as well as the set of lane line geometric parameters. The identification module is used to determine the reference lane line from the set of lane line geometric parameters and identify the spatial constraint pattern between the initial roadside observation point sequence and the reference lane line. The spatial constraint pattern includes at least the parallel constraint pattern and the convergence / separation constraint pattern. The evaluation module is used to evaluate the comprehensive confidence of the spatial constraint relationship of the current frame based on the time series feature vector constructed from the spatial constraint pattern of multiple consecutive frames within a preset window. The curb generation module is used to generate a set of discretized coordinate points of the final curb curve based on the comparison results of the observation confidence and the comprehensive confidence, using a preset routing strategy.

Images