A method and device for road edge detection based on a camera

Abstract

This invention discloses a road edge detection method and apparatus based on a camera. The method includes: acquiring road image data of a target area captured by the camera; extracting several road edge detection boxes from the road image data; and extracting road edge information based on a preset region of the several road edge detection boxes. The preset region includes at least one of a diagonal region, a vertically aligned central region parallel to the horizontal direction, and a horizontally aligned central region parallel to the vertical direction. This invention extracts road edge information from a preset region in the road image data with significant grayscale changes. This effectively reduces the influence of local information on road edge extraction. Even when road edges are indistinct or the road edge material is aged, road edge information is extracted stably and accurately, thereby effectively improving the road edge detection performance.

Description

Technical Field

[0001] This invention relates to the field of road detection technology, and in particular to a method and apparatus for road edge detection based on a camera. Background Technology

[0002] With the declining number of sanitation workers in my country, recruitment difficulties and an aging population are becoming increasingly severe, leading to a widening talent gap. Meanwhile, rapid urbanization is causing the demand for sanitation construction and maintenance to grow steadily each year. In this social context, autonomous driving technology, as an upgrade and supplement to the sanitation industry, has become an inevitable trend. For unmanned sanitation vehicles, edge detection is a crucial method to ensure operational effectiveness, and improving the detection of road edges is a key research topic in the development of unmanned sanitation vehicles. Summary of the Invention

[0003] This invention provides a road edge detection method and apparatus based on a camera to improve the detection effect of road edges.

[0004] An embodiment of the present invention provides a road edge detection method based on a camera, comprising:

[0005] Acquire road image data of the target area captured by the camera;

[0006] Extract several roadside detection boxes from the road image data, and extract roadside information based on a preset region of the several roadside detection boxes, wherein the preset region includes at least one of a diagonal region, a vertical center region parallel to the horizontal direction, and a horizontal center region parallel to the vertical direction.

[0007] Furthermore, the step of extracting several curb detection boxes from the road image data and extracting curb information based on a preset region of the several curb detection boxes specifically includes:

[0008] Extract several straight road edge detection boxes from the road image data, and extract the straight road edge according to the preset area of ​​the several straight road edge detection boxes;

[0009] Extract several curve edge detection boxes from the road image data, and extract the curve edge based on the preset area of ​​the several curve edge detection boxes.

[0010] Furthermore, the curb information also includes a fitted curve curb, and the method for obtaining the fitted curve curb includes:

[0011] Based on the positional relationship between the straight road edge and the curve road edge detection box, straight road edges to be fitted are selected from the straight road edges whose road edge endpoints are located in the same curve road edge detection box. The road edge endpoints of the straight road edges to be fitted are then fitted with curves to obtain the fitted curve road edge.

[0012] Furthermore, the step of extracting several curb detection boxes from the road image data and extracting curb information based on a preset region of the several curb detection boxes further includes:

[0013] Extract several virtual curb detection boxes from the road image data, and extract virtual curbs based on the preset areas of the virtual curb detection boxes.

[0014] Furthermore, the virtual road extends along the safety islands in the road.

[0015] Furthermore, after "extracting several curb detection boxes from the road image data, and extracting curb information based on a preset area of ​​the several curb detection boxes", the method further includes:

[0016] The various types of curb information are projected into a top view, and the actual curb where the vehicle is operating is selected based on the vehicle's working status.

[0017] Furthermore, the process of projecting various types of curb information into a top-down view and filtering out the actual curb where the vehicle is operating, in conjunction with the vehicle's working status, specifically involves:

[0018] Project the various types of curb information into a top view, and convert the pixel coordinates of the curb in the top view into actual world coordinates;

[0019] A rectangular frame with world coordinates is set in front of the vehicle. Based on the vehicle's working status, a curb that meets preset conditions is selected as the actual curb for the vehicle's operation, combining the rectangular frame and the top view.

[0020] Furthermore, the step of selecting a curb that meets preset conditions as the actual curb for vehicle operation based on the vehicle's working status, combined with the rectangular frame and the top view, specifically involves:

[0021] Project the top view onto the rectangular frame. If the vehicle is in the operation of obstacle avoidance, select the roadside that is farthest from the center point of the front of the vehicle in the top view as the actual roadside.

[0022] If the vehicle is in the edge-hugging operation mode, select the curb closest to the center point of the front of the vehicle in the top view. When the distance between the curb and the right side of the center point of the front of the vehicle is within a preset distance range, the curb is taken as the actual curb.

[0023] Furthermore, after "extracting several curb detection boxes from the road image data, and extracting curb information based on a preset area of ​​the several curb detection boxes", the method further includes:

[0024] The pixel coordinates of various roadside information are controlled in the road image data until a roadside appears in a preset area of ​​the road image data, and the roadside in the preset area is taken as the actual roadside for vehicle operation.

[0025] Another embodiment of the present invention provides a camera-based road edge detection device, comprising:

[0026] The image data acquisition module is used to acquire road image data of the target area captured by the camera;

[0027] The curb information extraction module is used to extract several curb detection boxes from the road image data, and extract curb information based on a preset area of ​​several curb detection boxes. The preset area includes one of a diagonal area, a vertical center area parallel to the horizontal direction, and a horizontal center area parallel to the vertical direction.

[0028] This invention, after extracting several curb detection boxes from road image data, extracts curb information based on a preset region of the curb detection boxes. The preset region includes at least one of a diagonal region, a vertically aligned central region parallel to the horizontal direction, or a horizontally aligned central region parallel to the vertical direction. This invention extracts curb information from preset regions with significant grayscale changes in road image data, effectively reducing the influence of local information on road edge extraction. Even when road edges are indistinct or the road edge material is aged, curb information is extracted stably and accurately, thus effectively improving curb detection performance. Furthermore, this invention filters out suitable straight curbs to be fitted based on the positional relationship between straight and curved curb detection boxes. The endpoints of these straight curbs are then fitted with curves to obtain fitted curved curbs. Compared to traditional techniques that segment the road surface and then fit edge lines, this effectively avoids noise introduced during segmentation, thereby improving the reliability of the fitted curved curbs.

[0029] Furthermore, in this embodiment of the invention, the acquired curb information is projected into a top view. The curb information includes straight curbs, curved curbs, virtual curbs, and fitted curved curbs. The top view projected from various curb information can comprehensively and accurately display the curb conditions on the actual road. Combining the top view with the vehicle's working status can accurately filter out the actual curb where the vehicle is operating. Attached Figure Description

[0030] Figure 1 This is a schematic flowchart of a road edge detection method based on a camera provided in an embodiment of the present invention;

[0031] Figure 2 This is a schematic diagram of a virtual curb provided in an embodiment of the present invention;

[0032] Figure 3 This is a schematic diagram of a camera-based road edge detection device provided in an embodiment of the present invention. Detailed Implementation

[0033] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0034] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "multiple" means two or more.

[0035] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0036] Please see Figure 1-2 In one embodiment of the present invention, a method is provided. Figure 1 The camera-based road edge detection method shown includes:

[0037] S1. Acquire road image data of the target area captured by the camera;

[0038] This invention can employ a single camera to acquire image data, or multiple cameras to acquire multiple image data sets. These multiple image data sets are then stitched together to obtain road image data that reflects the overall road conditions of the target area. The single camera can be a forward-looking short-focus camera or a forward-looking long-focus camera. This invention uses a forward-looking long-focus camera to acquire image data, which effectively increases the detection distance, resulting in more comprehensive road image data and thus significantly improving the accuracy of curb detection.

[0039] S2. Extract several roadside detection boxes from the road image data, and extract roadside information based on a preset area of ​​several roadside detection boxes. The preset area includes at least one of a diagonal area, a vertical center area parallel to the horizontal direction, and a horizontal center area parallel to the vertical direction.

[0040] Optionally, embodiments of the present invention can extract curb detection boxes from road image data using linear detection or deep learning neural networks. The curb detection boxes include straight curb detection boxes and curved curb detection boxes; correspondingly, the extracted curb information includes straight curbs and curved curbs. To improve the reliability of the curb detection boxes, embodiments of the present invention can also perform regression processing on the curb detection boxes to obtain regression boxes for further curb extraction. Regression processing methods include, but are not limited to, SSD, YOLO, and Faster R-CNN. In a specific implementation, embodiments of the present invention can perform cropping, quantization, and distillation optimization on the regression boxes of the detection boxes according to actual needs, thereby further improving the accuracy of curb extraction.

[0041] This invention extracts several curb detection boxes from image data and extracts curb information based on a preset area of ​​the curb detection boxes. During the curb extraction process, the significant grayscale changes at the diagonal corners of the road image are taken into account, and the local information of the edge is not sensitive. By extracting curb information based on the diagonal area of ​​the curb detection boxes, curb information can be extracted stably and accurately even in the presence of no obvious road edge or aging curb material, effectively improving the curb detection effect.

[0042] In one embodiment, after acquiring road image data, the road image data is preprocessed, such as by night enhancement, to improve the nighttime imaging effect.

[0043] In one embodiment, several roadside detection boxes are extracted from road image data, and roadside information is extracted based on a preset region of the several roadside detection boxes, specifically including:

[0044] Extract several straight road edge detection boxes from the road image data, and extract the straight road edge based on the preset area of ​​the several straight road edge detection boxes;

[0045] Extract several curve curb detection boxes from the road image data, and extract the curve curb based on the preset area of ​​the several curve curb detection boxes.

[0046] It is understood that curb information includes, but is not limited to, straight curbs and curved curbs. When extracting curbs based on the diagonal area of ​​the detection frame, this embodiment of the invention needs to distinguish between the upper left and lower right diagonal and the lower right and lower left diagonal of the detection frame, so as to realize the detection and extraction of different categories of curbs.

[0047] In one embodiment, the curb information further includes a fitted curve curb, and the method for obtaining the fitted curve curb includes:

[0048] Based on the positional relationship between the detection boxes of straight road edges and curved road edges, straight road edges to be fitted are selected from the straight road edges whose endpoints are located in the same curved road edge detection box. The endpoints of the straight road edges to be fitted are then fitted with curves to obtain the fitted curved road edges.

[0049] In one embodiment, based on the pixel relationship between the curved road edge detection box and the straight road edge in the image data, the positional relationship between the curved road edge detection box and the straight road edge can be obtained. This positional relationship is either an intersection relationship or an inclusion relationship. The method involves filtering out straight road edges whose endpoints are located within the same curved road edge detection box. Specifically: when the straight road edge and the curved road edge detection box are in an inclusion relationship (i.e., the curved road edge detection box includes the straight road edge), the straight road edge within the curved road edge detection box is taken as the straight road edge to be fitted; when the straight road edge and the curved road edge are in an intersection relationship, the straight road edge intersecting with the curved road edge detection box is taken as the straight road edge to be fitted, and the intersection point of the straight road edge to be fitted and the curved road edge detection box is taken as the endpoint of the straight road edge to be fitted. This embodiment of the invention obtains a fitted curved road edge by curve fitting the endpoint of the straight road edge to be fitted. Compared to the traditional method of segmenting the road surface from the image and then fitting the edge line, this effectively reduces noise during the segmentation process, thereby effectively improving the stability of the fitting.

[0050] In this embodiment of the invention, the fitted curve road edge is different from the curved road edge. The curved road edge is directly extracted from the road image data using the curve road edge detection box extraction method, while the fitted curve road edge is obtained by filtering the straight road edge to be fitted based on the positional relationship between the straight road edge and the curve road edge detection box, and then fitting it based on the road edge endpoints of the straight road edge to be fitted.

[0051] Curve fitting methods include, but are not limited to, polynomial fitting, linear interpolation fitting, bilinear interpolation fitting, and cubic spline interpolation fitting.

[0052] In one embodiment, extracting several roadside detection boxes from road image data and extracting roadside information based on a preset region of the several roadside detection boxes further includes:

[0053] Extract several virtual curb detection boxes from the road image data, and extract the virtual curb based on the preset area of ​​the virtual curb detection boxes.

[0054] Please see Figure 2A virtual curb is formed by extending from the safety island in the road. A rectangle is drawn between the safety island and the green belt along the road where the vehicle is located. The left edge of the safety island and the left edge of the road lie on the diagonal of this rectangle, and this diagonal is the virtual curb. It should be noted that a virtual curb refers to a curb where there is no clear dividing line on the ground in front of the vehicle, such as when there is no clear separation between concrete and asphalt road surfaces, a concrete road seam, or a significant difference in curb height.

[0055] In this embodiment of the invention, a rectangular frame is constructed by the safety island and the green belt of the road where the vehicle is located. The left edge of the safety island and the left edge of the road are located on the diagonal of the rectangle, and the diagonal of the rectangle is the extended virtual curb. The extraction of the virtual curb allows vehicles to work along the edge normally even when there is no obvious road boundary, thereby improving the reliability of edge-keeping operations.

[0056] In one embodiment, after "extracting several curb detection boxes from road image data and extracting curb information based on a preset area of ​​the several curb detection boxes", the method further includes:

[0057] Multiple types of curb information are projected into a top-down view, and the actual curb where the vehicle is operating is selected based on the vehicle's working status.

[0058] In this embodiment of the invention, various roadside information is projected into a top view, and the pixel coordinates of the roadside in the top view are converted into actual world coordinates. A rectangular frame with world coordinates is set in front of the vehicle. Based on the vehicle's working status, the roadside that meets the preset conditions is selected as the actual roadside for vehicle operation by combining the rectangular frame and the top view.

[0059] Optionally, in this embodiment of the invention, the camera needs to be calibrated before projecting various curbs to ensure that the coordinates are consistent after projection. The calibration method includes, but is not limited to, at least one of self-calibration, motion-based calibration, checkerboard calibration, and multi-sensor joint calibration.

[0060] In this embodiment of the invention, the coordinates of the curb and the rectangle in the top view are both unified world coordinates. This top view is projected onto the front of the vehicle. Assuming the vehicle is driving and operating on the road, the actual curb is selected from the top view within the rectangle based on the vehicle's current operating status and preset filtering conditions. The preset filtering conditions can be set according to actual needs.

[0061] In one embodiment, based on the vehicle's operating status, a curb that meets preset conditions is selected as the actual curb for vehicle operation, combining a rectangular frame and a top view. Specifically:

[0062] Project the top view onto the rectangular frame. If the vehicle is in obstacle avoidance mode, select the curb furthest from the center point of the front of the vehicle in the top view as the actual curb.

[0063] If the vehicle is in the edge-keeping operation mode, select the curb closest to the center point of the vehicle's front end in the top view. When the distance between the curb and the right side of the center point of the vehicle's front end is within the preset distance range, use the curb as the actual curb.

[0064] The preset distance range can be set according to actual operational needs. When the distance from the curb to the right of the center point of the front of the vehicle is too far, the curb is considered too short to be used as an actual curb.

[0065] The embodiments of the present invention combine the working status of the vehicle and select the appropriate curb as the actual curb under different selection conditions, which can effectively improve the working efficiency of the vehicle, such as improving the efficiency of sanitation vehicles in edge operation and obstacle avoidance operation.

[0066] In one embodiment, after "extracting several curb detection boxes from road image data, and extracting curb information based on a preset area of ​​the several curb detection boxes", the method further includes:

[0067] Multiple types of curb information are controlled by pixel coordinates in the road image data until a curb appears in a preset area of ​​the road image data. The curb in the preset area is then used as the actual curb for vehicle operation.

[0068] In this embodiment of the invention, the preset position can be determined based on the vehicle's driving mode, which is to drive on the right. Based on the vehicle's driving direction, a certain position on the right side of the driving direction is used as the preset area in the road image data. This embodiment of the invention uses coordinate control to ensure that the curb appears in the preset area, thereby enabling the quick and convenient determination of the actual curb where the vehicle is operating.

[0069] Implementing the embodiments of the present invention has the following beneficial effects:

[0070] This invention, after extracting several curb detection boxes from road image data, extracts curb information based on a preset region of the curb detection boxes. The preset region includes at least one of a diagonal region, a vertically aligned central region parallel to the horizontal direction, or a horizontally aligned central region parallel to the vertical direction. This invention extracts curb information from preset regions with significant grayscale changes in road image data, effectively reducing the influence of local information on road edge extraction. Even when road edges are indistinct or the road edge material is aged, curb information is extracted stably and accurately, thus effectively improving curb detection performance. Furthermore, this invention filters out suitable straight curbs to be fitted based on the positional relationship between straight and curved curb detection boxes. The endpoints of these straight curbs are then fitted with curves to obtain fitted curved curbs. Compared to traditional techniques that segment the road surface and then fit edge lines, this effectively avoids noise introduced during segmentation, thereby improving the reliability of the fitted curved curbs.

[0071] Furthermore, in this embodiment of the invention, the acquired curb information is projected into a top view. The curb information includes straight curbs, curved curbs, virtual curbs, and fitted curved curbs. The top view projected from various curb information can comprehensively and accurately display the curb conditions on the actual road. Combining the top view with the vehicle's working status can accurately filter out the actual curb where the vehicle is operating.

[0072] Please see Figure 3 One embodiment of the present invention provides a camera-based road edge detection device, comprising:

[0073] Image data acquisition module 10 is used to acquire road image data of the target area captured by the camera;

[0074] Optionally, embodiments of the present invention may use a single camera to collect image data, or multiple cameras to collect multiple image data, and stitch the multiple image data together to obtain road image data that can reflect the overall road conditions of the target area. The single camera may be a forward-looking short-focus camera or a forward-looking long-focus camera. The forward-looking long-focus camera can effectively increase the detection distance, making the collected road image data more comprehensive, thereby effectively improving the accuracy of curb detection.

[0075] The curb information extraction module 20 is used to extract several curb detection boxes from the road image data and extract curb information based on a preset area of ​​several curb detection boxes. The preset area includes at least one of a diagonal area, a vertical center area parallel to the horizontal direction, and a horizontal center area parallel to the vertical direction.

[0076] Optionally, embodiments of the present invention can extract curb detection boxes from road image data using linear detection or deep learning neural networks. The curb detection boxes include straight curb detection boxes and curved curb detection boxes; correspondingly, the extracted curb information includes straight curbs and curved curbs. To improve the reliability of the curb detection boxes, embodiments of the present invention can also perform regression processing on the curb detection boxes to obtain regression boxes for further curb extraction. Regression processing methods include, but are not limited to, SSD, YOLO, and Faster R-CNN. In a specific implementation, embodiments of the present invention can perform cropping, quantization, and distillation optimization on the regression boxes of the detection boxes according to actual needs, thereby further improving the accuracy of curb extraction.

[0077] This invention extracts several curb detection boxes from image data and extracts curb information based on a preset area of ​​the curb detection boxes. During the curb extraction process, the significant grayscale changes at the diagonal corners of the road image are taken into account, and the local information of the edge is not sensitive. By extracting curb information based on the diagonal area of ​​the curb detection boxes, curb information can be extracted stably and accurately even in the presence of no obvious road edge or aging curb material, effectively improving the curb detection effect.

[0078] In one embodiment, the curb information extraction module 20 includes components for:

[0079] Extract several straight road edge detection boxes from the road image data, and extract the straight road edge based on the preset area of ​​the several straight road edge detection boxes;

[0080] Extract several curve curb detection boxes from the road image data, and extract the curve curb based on the preset area of ​​the several curve curb detection boxes.

[0081] It is understood that curb information includes, but is not limited to, straight curbs and curved curbs. When extracting curbs based on the diagonal area of ​​the detection frame, this embodiment of the invention needs to distinguish between the upper left and lower right diagonal and the lower right and lower left diagonal of the detection frame, so as to realize the detection and extraction of different categories of curbs.

[0082] In one embodiment, the curb information further includes a fitted curve curb, and the method for obtaining the fitted curve curb includes:

[0083] Based on the positional relationship between the detection boxes of straight road edges and curved road edges, straight road edges to be fitted are selected from the straight road edges whose endpoints are located in the same curved road edge detection box. The endpoints of the straight road edges to be fitted are then fitted with curves to obtain the fitted curved road edges.

[0084] In one embodiment, based on the pixel relationship between the curved road edge detection box and the straight road edge in the image data, the positional relationship between the curved road edge detection box and the straight road edge can be obtained. This positional relationship is either an intersection relationship or an inclusion relationship. The method involves filtering out straight road edges whose endpoints are located within the same curved road edge detection box. Specifically: when the straight road edge and the curved road edge detection box are in an inclusion relationship (i.e., the curved road edge detection box includes the straight road edge), the straight road edge within the curved road edge detection box is taken as the straight road edge to be fitted; when the straight road edge and the curved road edge are in an intersection relationship, the straight road edge intersecting with the curved road edge detection box is taken as the straight road edge to be fitted, and the intersection point of the straight road edge to be fitted and the curved road edge detection box is taken as the endpoint of the straight road edge to be fitted. This embodiment of the invention obtains a fitted curved road edge by curve fitting the endpoint of the straight road edge to be fitted. Compared to the traditional method of segmenting the road surface from the image and then fitting the edge line, this effectively reduces noise during the segmentation process, thereby effectively improving the stability of the fitting.

[0085] In this embodiment of the invention, the fitted curve road edge is different from the curved road edge. The curved road edge is directly extracted from the road image data using the curve road edge detection box extraction method, while the fitted curve road edge is obtained by filtering the straight road edge to be fitted based on the positional relationship between the straight road edge and the curve road edge detection box, and then fitting it based on the road edge endpoints of the straight road edge to be fitted.

[0086] Curve fitting methods include, but are not limited to, polynomial fitting, linear interpolation fitting, bilinear interpolation fitting, and cubic spline interpolation fitting.

[0087] In one embodiment, the curb information extraction module 20 is further configured to:

[0088] Extract several virtual curb detection boxes from the road image data, and extract the virtual curb based on the preset area of ​​the virtual curb detection boxes.

[0089] Please see Figure 2 A virtual curb is formed by extending from the safety island in the road. A rectangle is drawn between the safety island and the green belt along the road where the vehicle is located. The left edge of the safety island and the left edge of the road lie on the diagonal of this rectangle, and this diagonal is the virtual curb. It should be noted that a virtual curb refers to a curb where there is no clear dividing line on the ground in front of the vehicle, such as when there is no clear separation between concrete and asphalt road surfaces, a concrete road seam, or a significant difference in curb height.

[0090] In this embodiment of the invention, a rectangular frame is constructed by the safety island and the green belt of the road where the vehicle is located. The left edge of the safety island and the left edge of the road are located on the diagonal of the rectangle, and the diagonal of the rectangle is the extended virtual curb. The extraction of the virtual curb allows vehicles to work along the edge normally even when there is no obvious road boundary, thereby improving the reliability of edge-keeping operations.

[0091] In one embodiment, an actual curb screening module is also included, for:

[0092] Multiple types of curb information are projected into a top-down view, and the actual curb where the vehicle is operating is selected based on the vehicle's working status.

[0093] In this embodiment of the invention, various roadside information is projected into a top view, and the pixel coordinates of the roadside in the top view are converted into actual world coordinates. A rectangular frame with world coordinates is set in front of the vehicle. Based on the vehicle's working status, the roadside that meets the preset conditions is selected as the actual roadside for vehicle operation by combining the rectangular frame and the top view.

[0094] Optionally, in this embodiment of the invention, the camera needs to be calibrated before projecting various curbs to ensure that the coordinates are consistent after projection. The calibration method includes, but is not limited to, at least one of self-calibration, motion-based calibration, checkerboard calibration, and multi-sensor joint calibration.

[0095] In this embodiment of the invention, the coordinates of the curb and the rectangle in the top view are both unified world coordinates. This top view is projected onto the front of the vehicle. Assuming the vehicle is driving and operating on the road, the actual curb is selected from the top view within the rectangle based on the vehicle's current operating status and preset filtering conditions. The preset filtering conditions can be set according to actual needs.

[0096] In one embodiment, the actual curb screening module is specifically used for:

[0097] Project the top view onto the rectangular frame. If the vehicle is in obstacle avoidance mode, select the curb furthest from the center point of the front of the vehicle in the top view as the actual curb.

[0098] If the vehicle is in the edge-keeping operation mode, select the curb closest to the center point of the vehicle's front end in the top view. When the distance between the curb and the right side of the center point of the vehicle's front end is within the preset distance range, use the curb as the actual curb.

[0099] The preset distance range can be set according to actual operational needs. When the distance from the curb to the right side of the center point of the front of the vehicle is too far, it is determined that the length of the curb is too short to be used as the actual curb.

[0100] The embodiments of the present invention combine the working status of the vehicle and select the appropriate curb as the actual curb under different selection conditions, which can effectively improve the working efficiency of the vehicle, such as improving the efficiency of sanitation vehicles in edge operation and obstacle avoidance operation.

[0101] In one embodiment, the actual curb screening module is also used for:

[0102] Multiple types of curb information are controlled by pixel coordinates in the road image data until a curb appears in a preset area of ​​the road image data. The curb in the preset area is then used as the actual curb for vehicle operation.

[0103] In this embodiment of the invention, the preset position can be determined based on the vehicle's driving mode, which is to drive on the right. Based on the vehicle's driving direction, a certain position on the right side of the driving direction is used as the preset area in the road image data. This embodiment of the invention uses coordinate control to ensure that the curb appears in the preset area, thereby enabling the quick and convenient determination of the actual curb where the vehicle is operating.

[0104] Implementing the embodiments of the present invention has the following beneficial effects:

[0105] This invention, after extracting several curb detection boxes from road image data, extracts curb information based on a preset region of the curb detection boxes. The preset region includes at least one of a diagonal region, a vertically aligned central region parallel to the horizontal direction, or a horizontally aligned central region parallel to the vertical direction. This invention extracts curb information from preset regions with significant grayscale changes in road image data, effectively reducing the influence of local information on road edge extraction. Even when road edges are indistinct or the road edge material is aged, curb information is extracted stably and accurately, thus effectively improving curb detection performance. Furthermore, this invention filters out suitable straight curbs to be fitted based on the positional relationship between straight and curved curb detection boxes. The endpoints of these straight curbs are then fitted with curves to obtain fitted curved curbs. Compared to traditional techniques that segment the road surface and then fit edge lines, this effectively avoids noise introduced during segmentation, thereby improving the reliability of the fitted curved curbs.

[0106] Furthermore, in this embodiment of the invention, the acquired curb information is projected into a top view. The curb information includes straight curbs, curved curbs, virtual curbs, and fitted curved curbs. The top view projected from various curb information can comprehensively and accurately display the curb conditions on the actual road. Combining the top view with the vehicle's working status can accurately filter out the actual curb where the vehicle is operating.

[0107] The above are preferred embodiments of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A road edge detection method based on a camera, characterized in that, include: Acquire road image data of the target area captured by the camera; Extract several road edge detection boxes from the road image data, and extract road edge information based on a preset region of several road edge detection boxes, wherein the preset region includes at least one of a diagonal region, a vertical center region parallel to the horizontal direction, and a horizontal center region parallel to the vertical direction; The step of extracting several roadside detection boxes from the road image data and extracting roadside information based on a preset region of the several roadside detection boxes specifically includes: Extract several straight road edge detection boxes from the road image data, and extract the straight road edge according to the preset area of ​​the several straight road edge detection boxes; Extract several curve edge detection boxes from the road image data, and extract the curve edge according to the preset area of ​​the several curve edge detection boxes; Based on the positional relationship between the straight road edge and the curve road edge detection box, straight road edges to be fitted are selected from the straight road edges whose road edge endpoints are located in the same curve road edge detection box. The road edge endpoints of the straight road edges to be fitted are then fitted with curves to obtain the fitted curve road edges.

2. The road edge detection method based on a camera as described in claim 1, characterized in that, The step of extracting several roadside detection boxes from the road image data and extracting roadside information based on a preset region of the several roadside detection boxes further includes: Extract several virtual curb detection boxes from the road image data, and extract virtual curbs based on the preset areas of the virtual curb detection boxes.

3. The road edge detection method based on a camera as described in claim 2, characterized in that, The virtual curb is formed by extending from the safety islands in the road.

4. The road edge detection method based on a camera as described in claim 1, characterized in that, After "extracting several curb detection boxes from the road image data, and extracting curb information based on a preset area of ​​the several curb detection boxes", the process also includes: The various types of curb information are projected into a top view, and the actual curb where the vehicle is operating is selected based on the vehicle's working status.

5. The road edge detection method based on a camera as described in claim 4, characterized in that, The process of projecting various types of curb information into a top-down view and filtering out the actual curb where the vehicle is operating, specifically involves: Project the various types of curb information into a top view, and convert the pixel coordinates of the curb in the top view into actual world coordinates; A rectangular frame with world coordinates is set in front of the vehicle. Based on the vehicle's working status, a curb that meets preset conditions is selected as the actual curb for the vehicle's operation, combining the rectangular frame and the top view.

6. The road edge detection method based on a camera as described in claim 5, characterized in that, The step of selecting a curb that meets preset conditions as the actual curb for vehicle operation based on the vehicle's working status, combined with the rectangular frame and the top view, specifically involves: Project the top view onto the rectangular frame; If the vehicle is operating in obstacle avoidance mode, select the curb furthest from the center point of the front of the vehicle in the top view as the actual curb; or, if the vehicle is operating in edge-hugging mode, select the curb closest to the center point of the front of the vehicle in the top view, and when the distance between the curb and the right side of the center point of the front of the vehicle is within a preset distance range, use the curb as the actual curb.

7. The road edge detection method based on a camera as described in claim 1, characterized in that, After "extracting several curb detection boxes from the road image data, and extracting curb information based on a preset area of ​​the several curb detection boxes", the process also includes: The pixel coordinates of various roadside information are controlled in the road image data until a roadside appears in a preset area of ​​the road image data, and the roadside in the preset area is taken as the actual roadside for vehicle operation.

8. A camera-based road edge detection device, characterized in that, include: A control module configured to perform the camera-based road edge detection method according to any one of claims 1 to 7.

Images