A method, device, vehicle and storage medium for detecting boundary lines of a drivable area

The image is captured by a fisheye camera and divided image blocks, and the line segments are fitted with the regression method, which solves the problem of large amount of boundary line detection in the prior art, and achieves a larger detection range and more efficient boundary line detection.

CN115063763BActive Publication Date: 2024-11-15GUANGZHOU WERIDE TECH LTD CO
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210569986.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-05-24
Publication Date
2024-11-15
Estimated Expiration
2042-05-24

AI Technical Summary

Technical Problem

In the prior art, the detection and calculation of the boundary line of the feasible area is large, resulting in inefficiency.

Method used

The image is captured by a fisheye camera, and by dividing image blocks and fitting line segments using a regression method, the calculation amount of the network model is reduced, and the second preset network model is used to determine whether there are line segments in the image, reducing unnecessary processing.

Benefits of technology

It improves the detection range and efficiency, reduces the complexity of curve fitting, reduces the calculation amount of network models, and improves the boundary line detection performance of the autonomous driving assistance system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115063763B_ABST
    Figure CN115063763B_ABST
Patent Text Reader

Abstract

The present invention discloses a method, device, vehicle and storage medium for detecting the boundary line of a drivable area. The method comprises: dividing a first image according to a specified range to obtain a plurality of second images; processing the second image using a first preset network model to obtain a line segment representation equation of the second image; sampling key points of the second image according to the line segment representation equation, connecting the key points in the second image to obtain a third image; traversing the third images with adjacent positions, connecting the closest key points between the third images with adjacent positions, and obtaining the boundary line of the drivable area. Through the embodiments of the present invention, a picture taken by a fisheye camera is used as a model input to obtain a larger detection range; the image block division method is used to reduce the complexity of curve fitting; and the regression method is used to fit the line segment to reduce the calculation amount of the network model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of vehicles, and in particular to a method and device for detecting a boundary line of a drivable area, a vehicle and a storage medium. Background Art

[0002] In the automated driving assistance system, the boundary line detection of the drivable area is one of the important tasks. The demarcation of the drivable area helps the automated driving assistance system to plan and make decisions on the road. Existing methods usually use deep learning segmentation methods to obtain the boundary line of the drivable area. The segmentation method obtains the boundary line of the drivable area, which has the problem of large amount of calculation. Summary of the invention

[0003] The main purpose of the present invention is to provide a method, device, vehicle and storage medium for detecting the boundary line of a drivable area, aiming to solve the problem of large amount of calculation in the prior art of detecting the boundary line of a drivable area.

[0004] To achieve the above object, the present invention provides a method for detecting a drivable area boundary line, the method comprising the following steps:

[0005] S1: Divide the first image according to a specified range to obtain a plurality of second images;

[0006] S2: Processing the second image using the first preset network model to obtain a line segment representation equation of the second image;

[0007] S3: sampling key points of the second image according to the line segment representation equation, connecting the key points in the second image to obtain a third image;

[0008] S4: traversing the third images that are adjacent in position, connecting the closest key points between the third images that are adjacent in position, and obtaining a boundary line of the drivable area.

[0009] Optionally, the first preset network model is trained by the following steps:

[0010] dividing the fourth image according to a specified range to obtain a plurality of fifth images;

[0011] Processing the fifth image to obtain a line segment representation equation of the fifth image;

[0012] The first preset network model is trained using the fifth image and the line segment representation equation.

[0013] Optionally, before step S2, the following steps are also included:

[0014] Processing the second image using a second preset network model to determine whether a line segment exists in the second image;

[0015] The second image with line segments is provided to the first preset network model for processing.

[0016] Optionally, the performing key point sampling on the second image according to the line segment representation equation comprises the following steps:

[0017] generating a line segment in the second image according to the line segment representation equation;

[0018] The line segment is sampled according to a specified length to obtain a key point; or, the line segment is sampled using a third preset network model to obtain a key point.

[0019] Optionally, after step S1, the following steps are further included:

[0020] Coordinates are set for each of the second images, where the coordinates are used to identify the position of the second image in the coordinate system.

[0021] Optionally, the first image is obtained by photographing a camera with a wide-angle photographing function.

[0022] In addition, to achieve the above-mentioned purpose, the present invention also proposes a drivable area boundary line detection device, the device comprising:

[0023] An image division unit, used for dividing the first image according to a specified range to obtain a plurality of second images;

[0024] an equation acquisition unit, configured to process the second image using a first preset network model to acquire a line segment representation equation of the second image;

[0025] A key point sampling unit, configured to perform key point sampling on the second image according to the line segment representation equation, and connect the key points in the second image to obtain a third image;

[0026] The key point connection unit is used to traverse the adjacent third images, connect the closest key points of the adjacent third images, and obtain the boundary line of the drivable area.

[0027] Optionally, the device further comprises:

[0028] The line segment judgment unit is used to process the second image using the second preset network model to determine whether there are line segments in the second image; and provide the second image with line segments to the first preset network model for processing.

[0029] In addition, to achieve the above-mentioned purpose, the present invention also proposes a vehicle, which includes: a memory, a processor, and a drivable area boundary line detection program stored in the memory and runnable on the processor, and the drivable area boundary line detection program is configured to implement the steps of the drivable area boundary line detection method described above.

[0030] In addition, to achieve the above-mentioned purpose, the present invention also proposes a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the drivable area boundary line detection method as described above are implemented.

[0031] Through the embodiments of the present invention, pictures taken by a fisheye camera are used as model input to obtain a larger detection range; the image block division method is used to reduce the complexity of curve fitting; and the regression method is used to fit line segments to reduce the amount of calculation of the network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A schematic flow chart of the drivable area boundary line detection method provided by the present invention.

[0033] Figure 2 A schematic diagram of a flow chart for training a first preset network model provided by the present invention.

[0034] Figure 3 Another schematic flow chart of the drivable area boundary line detection method provided by the present invention.

[0035] Figure 4 A schematic diagram of the key point sampling process provided by the present invention.

[0036] Figure 5 This is a structural block diagram of an embodiment of the drivable area boundary line detection device provided by the present invention.

[0037] Figure 6 This is another structural block diagram of an embodiment of the drivable area boundary line detection device provided by the present invention.

[0038] Figure 7 It is a schematic diagram of the structure of a transportation tool of the hardware operating environment involved in the embodiment of the present invention.

[0039] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0040] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention clearer and more understandable, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0041] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present invention, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0042] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0043] In one embodiment, Figure 1 As shown, the present invention provides a method for detecting a drivable area boundary line, the method comprising:

[0044] Step S1, dividing a first image according to a specified range to obtain a plurality of second images.

[0045] A shooting device with a fisheye lens (such as a fisheye camera) is installed on the unmanned vehicle to shoot the road surface information during the driving process of the vehicle to obtain a road surface information picture. By shooting images with a fisheye camera, image information of a larger range can be obtained. Due to the distortion and complexity of the boundary lines of images shot with a fisheye camera, it is necessary to divide the captured images into certain sizes. For example, the captured images are divided into 512×512 (pixels) to obtain multiple divided pictures. The specific division size can be set according to actual needs.

[0046] The coordinates of each divided second image are set, and the coordinates are used to identify the position of the second image in the coordinate system. The divided image has a unique number, which records the order of the divided images. The position of the divided image before division (such as the position in the camera coordinate system) is identified by the X and Y axis coordinates, as shown in the following table:

[0047] image X-axis number Y-axis serial number Image A 1 1 Image B 2 2 Image C 3 3 Image D 4 4 Image E 5 5

[0048] When performing boundary line detection, it is necessary to obtain image information around the vehicle. Therefore, the larger the range around the vehicle, the more information is obtained, and the more boundary lines are detected. Therefore, it is necessary to use a shooting device with a wide-angle shooting function to shoot images, so as to obtain image information within a larger geographical range, such as using a fisheye camera for shooting.

[0049] In addition to using a fisheye camera for shooting, this technical solution can also use other shooting devices for shooting, such as a wide-angle camera. The specific shooting device used is not limited by this technical solution.

[0050] Step S2: Use the first preset network model to process the second image to obtain a line segment representation equation of the second image.

[0051] The first preset network module is a deep learning neural network, or it can be other neural networks. The specific neural network used is not limited in this technical solution. The core is to use the trajectory information in the second image collected by the nonlinear fitting of the neural network, and use the gradient descent method to enable the neural network to have the ability to judge the correspondence between the trajectory and the representation equation, that is, the neural network can output the corresponding representation equation according to the input trajectory after training.

[0052] The expression equation corresponding to the trajectory is obtained according to the specific trajectory. The expression equation can be a curve fitting equation: r = x*con(theta) + y*sin(theta); a straight line equation: y = kx + b; a second-order polynomial: y = a + b*x + c*x^2, etc.

[0053] The first preset network adopts the supervised model training method. For the specific training process, see Figure 2 The process described.

[0054] Step S101: Divide the fourth image according to a specified range to obtain a plurality of fifth images.

[0055] A shooting device with a fisheye lens (such as a fisheye camera) is installed on the unmanned vehicle to shoot the road information during the driving process of the vehicle to obtain a road information picture. By shooting images with a fisheye camera, image information of a larger range can be obtained. Since the images shot with a fisheye camera are distorted and the boundary lines are inherently complex, it is necessary to divide the captured images and divide the images according to a certain size. For example, the captured images are divided according to 512×512 (pixels) to obtain multiple divided pictures.

[0056] Step S102: Process the fifth image to obtain a line segment representation equation of the fifth image.

[0057] The segmented image is processed using the Hough transform to obtain the representation equation of the curve in the image. The Hough transform is a feature extraction technology in image processing and computer vision. It uses the transformation of the two coordinate spaces, image space and Hough space, to map curves or straight lines with the same shape in one space to points in another coordinate space to vote and form peaks, thereby converting the problem of detecting arbitrary shapes into a statistical peak problem. The most important feature of the Hough transform is that it has good robustness to noise or incomplete shapes due to the introduction of probabilistic peak statistics.

[0058] After each image containing the boundary of the drivable area is processed by Hough transform, a representation equation can be obtained. As shown in the following table:

[0059] Serial number Fifth Image Representing equations 1 Image A Equation A 2 Image B Equation B 3 Image C Equation C 4 Image D Equation D 5 Image E Equation E

[0060] Detecting the image by Hough transform and obtaining the corresponding expression equation of the line belongs to the existing technical solution, and this solution will not be described in detail. Other technical solutions can also be used to detect the image and obtain the corresponding expression equation of the line; the present invention does not specifically limit which technical solution is used.

[0061] Step S103: Use the fifth image and the line segment representation equation to train the first preset network model.

[0062] The fifth image refers to an image used to train the first pre-network model. In this embodiment, the first pre-network model is a deep learning network model. In the embodiment of the present application, a supervised model training method is adopted, so the sample image has a label, which is used to label the real representation equation of the sample image. In the embodiment of the present application, the sample image and the representation equation are shown in the following table:

[0063] Serial number Fifth Image Representing equations 1 Image A Equation A 2 Image B Equation B 3 Image C Equation C 4 Image D Equation D 5 Image E Equation E

[0064] The first pre-network modeling can use different neural networks, or a large network with multiple branches for regression, that is, each branch can be trained independently and the operations are independent of each other. The core is to use the line segment information collected by the nonlinear fitting of the neural network, and use the gradient descent method to enable the neural network to learn the relationship between the line segment and the expression equation.

[0065] Use the trained deep learning network to process the segmented image containing the boundary line to obtain the representation equation corresponding to the boundary line. The corresponding relationship between the image and the representation equation is shown in the following table:

[0066] Serial number Second Image Representing equations 1 Image G Equation A 2 Image H Equation A 3 Image I Equation D 4 Image G Equation B 5 Image K Equation C

[0067] Step S3: sampling key points of the second image according to the line segment representation equation, connecting the key points in the second image to obtain a third image.

[0068] The key points on the line segment in the image are sampled using the representation equation (such as equation A) corresponding to the segmented second image (such as image G). When sampling, a key point can be sampled at a certain distance, or special points on the line segment can be sampled, such as points where the sampled line segment is bent. The specific sampling scheme is not limited by this technical solution.

[0069] The sampled key points are connected to obtain connecting line segments containing the key points.

[0070] Step S4, traversing the third images adjacent to each other, connecting the closest key points between the third images adjacent to each other, and obtaining a boundary line of the drivable area.

[0071] The captured image is divided into a certain size (e.g., the captured image is divided into 512×512 (pixels)), and multiple divided images are obtained. The divided images have a unique number, which records the order of the divided images. The position of the divided images before the division can be identified by the X and Y axis coordinates, as shown in the following table:

[0072]

[0073]

[0074] For the image after key point sampling, find the adjacent image through the X and Y axis serial numbers of the image. For example, if the image adjacent to image A is image B, then connect the key points in image A with the key points in image B. When connecting, connect the key points in image A that are closest to image B, so as to obtain the line segment after the key points between image A and image B are connected. Similarly, connect the key points in all the segmented second images to obtain the boundary line of the drivable area, such as the boundary line of the lane line and the curb.

[0075] In this embodiment, the key point sampling of step S3 is performed as follows: Figure 4 The process shown is used for sampling.

[0076] Step S201: Generate a line segment in the second image according to the line segment representation equation.

[0077] According to the representation equation obtained in step S2 (such as equation A corresponding to image H), curve B is drawn in image H using equation A.

[0078] Step S202: Sampling the line segment according to the specified length to obtain key points.

[0079] According to a certain length, such as 100 pixels, a key point is taken, thereby obtaining multiple key points on the curve. For example, for curve B, a key point is taken every 100 pixels, thereby obtaining multiple key points.

[0080] Step S203: Use the third preset network model to sample the line segment to obtain key points.

[0081] The curve corresponding to the expression equation is processed using the third preset network model to obtain key points in the curve.

[0082] The third preset network model can be a deep learning network model or other network models. In the embodiment of the present application, the third preset network model is a deep learning network model, which adopts a supervised model training method, so the sample image has annotations, and the key points of the sample image are annotated. In the embodiment of the present application, the sample images and key points are shown in the following table:

[0083] Serial number Line segment image Key Points 1 Line segment image A Key point A, key point B, key point C 2 Line segment image B Key point A, key point F, key point G 3 Line segment image C Key point G, key point B, key point H 4 Line segment image D Key point G, key point F, key point H

[0084] By training the deep learning network model, the network model can output the key points of the curve according to the input curve.

[0085] In the embodiment, only one of the key point sampling methods of step S202 and step S203 needs to be used, such as using the key point sampling method of step S202 or step S203 for sampling.

[0086] Through the embodiments of the present invention, pictures taken by a fisheye camera are used as model input to obtain a larger detection range; the image block division method is used to reduce the complexity of curve fitting; and the regression method is used to fit line segments to reduce the amount of calculation of the network model.

[0087] In addition, the present invention also provides a method for detecting the boundary line of a drivable area. Figure 3 The method is Figure 1 Based on the method shown, the following process is also included:

[0088] Step S5: Use the second preset network model to process the second image to determine whether there are line segments in the second image; and provide the second image with line segments to the first preset network model for processing.

[0089] The second image after segmentation is processed using the second preset network model to determine whether there are line segments in the second image. If there are line segments, the line segment features are extracted (such as lane line features). Then, the image containing only line segments (line segment features are extracted by the second preset network, and other features are not extracted) is provided to the first preset network model for processing, and the image not containing line segments does not need to be provided to the first preset network model for information processing. This reduces the amount of calculation of the first preset network model.

[0090] The second preset network model may be various types of network models capable of extracting image features. The embodiment of the present application does not limit the structure of the second preset network model and the extracted image features. For example, the second preset network model may refer to a U-Net (U-Network) model, a PSPNet (Pyramid Scene Parsing Network) model, a DenseNet (Dense Convolutional Network) model, a ResNet (Residual Network) model, or a MobileNet (Mobile Network) model, etc. The second preset network model itself may have initial parameters, which may be parameters obtained by pre-training on the ImageNet data set. During the first iterative training process, the second preset network model is trained based on the initial parameters.

[0091] The second preset network model is used to identify and semantically segment the second image. Before training the second preset network model, the parameters of the second preset network model can be initialized. For example, the parameters of the second preset network model are initialized using a Gaussian distribution with a variance of 0.01 and a mean of 0. For another example, the parameters of the second preset network model are initialized using a Gaussian distribution with a variance of 0.001 and a mean of 0. Different model parameters can be initialized in the same way or in different ways, which is not limited in the embodiments of the present application. The structure and parameter type of the second preset network model can be flexibly set according to demand, which is not limited in the embodiments of the present application. After setting the structure and parameter type and initializing the parameters, the second preset network model is obtained.

[0092] In this embodiment, the second preset network model is trained in a labeled manner, such as labeling a picture with lane lines and curbs, and labeling the presence of line segments and semantic features in the image.

[0093] The obtained boundary line is projected from the two-dimensional coordinate system to the three-dimensional coordinate system. The specific projection method used is not limited in this embodiment. For example, the boundary line can be projected from the two-dimensional coordinate system to the three-dimensional coordinate system according to the ground equation.

[0094] Projecting from a two-dimensional coordinate system to a three-dimensional coordinate system belongs to the existing technical solution and will not be described in detail in this embodiment.

[0095] Through the embodiment of the present invention, the second preset network is used to determine whether there are line segments in the segmented image, and then the image containing the line segments and the line segment features are extracted is provided to the first preset network for processing, thereby reducing the images provided to the first preset network for processing and reducing the calculation amount of the network model.

[0096] In addition, the embodiment of the present invention also provides a drivable area boundary line detection device, referring to Figure 5 , the drivable area boundary line detection device comprises:

[0097] An image dividing unit 10, configured to divide the first image into specified ranges to obtain a plurality of second images;

[0098] An equation acquisition unit 20, configured to process the second image using a first preset network model to acquire a line segment representation equation of the second image;

[0099] A key point sampling unit 30, configured to perform key point sampling on the second image according to the line segment representation equation, and connect the key points in the second image to obtain a third image;

[0100] The key point connecting unit 40 is used to traverse the adjacent third images, connect the closest key points of the adjacent third images, and obtain the boundary line of the drivable area.

[0101] Through the embodiments of the present invention, pictures taken by a fisheye camera are used as model input to obtain a larger detection range; the image block division method is used to reduce the complexity of curve fitting; and the regression method is used to fit line segments to reduce the amount of calculation of the network model.

[0102] In addition, the embodiment of the present invention also provides a drivable area boundary line detection device, referring to Figure 6 .exist Figure 5 Based on the drivable area boundary line detection device shown in the figure, it also includes:

[0103] The line segment judgment unit 50 is used to process the second image using the second preset network model to determine whether there are line segments in the second image; and provide the second image with line segments to the first preset network model for processing.

[0104] Through the embodiment of the present invention, the second preset network is used to determine whether there are line segments in the segmented image, and then the image containing the line segments and the line segment features are extracted is provided to the first preset network for processing, thereby reducing the images provided to the first preset network for processing and reducing the calculation amount of the network model.

[0105] Reference Figure 7 , Figure 7 A schematic diagram of the structure of a vehicle in the hardware operating environment involved in an embodiment of the present invention.

[0106] like Figure 7 As shown, the vehicle may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as WI-FI, 4G, 5G interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0107] Those skilled in the art will understand that Figure 7 The structure shown in the figure does not constitute a limitation on the vehicle, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0108] like Figure 7 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a drivable area boundary line detection program.

[0109] exist Figure 7 In the vehicle shown, the network interface 1004 is mainly used for data communication with an external network; the user interface 1003 is mainly used for receiving user input instructions; the vehicle calls the drivable area boundary line detection program stored in the memory 1005 through the processor 1001, and performs the following operations:

[0110] S1: Divide the first image according to a specified range to obtain a plurality of second images;

[0111] S2: Processing the second image using the first preset network model to obtain a line segment representation equation of the second image;

[0112] S3: sampling key points of the second image according to the line segment representation equation, connecting the key points in the second image to obtain a third image;

[0113] S4: traversing the third images that are adjacent in position, connecting the closest key points between the third images that are adjacent in position, and obtaining a boundary line of the drivable area.

[0114] Optionally, the first preset network model is trained by the following steps:

[0115] dividing the fourth image according to a specified range to obtain a plurality of fifth images;

[0116] Processing the fifth image to obtain a line segment representation equation of the fifth image;

[0117] The first preset network model is trained using the fifth image and the line segment representation equation.

[0118] Optionally, before step S2, the following steps are also included:

[0119] Processing the second image using a second preset network model to determine whether a line segment exists in the second image;

[0120] The second image with line segments is provided to the first preset network model for processing.

[0121] Optionally, the performing key point sampling on the second image according to the line segment representation equation comprises the following steps:

[0122] generating a line segment in the second image according to the line segment representation equation;

[0123] The line segment is sampled according to a specified length to obtain a key point; or, the line segment is sampled using a third preset network model to obtain a key point.

[0124] Optionally, after step S1, the following steps are further included:

[0125] Coordinates are set for each of the second images, where the coordinates are used to identify the position of the second image in the coordinate system.

[0126] Optionally, the first image is obtained by photographing a camera with a wide-angle photographing function.

[0127] Through the embodiments of the present invention, pictures taken by a fisheye camera are used as model input to obtain a larger detection range; the image block division method is used to reduce the complexity of curve fitting; and the regression method is used to fit line segments to reduce the amount of calculation of the network model.

[0128] In addition, an embodiment of the present invention further provides a computer-readable storage medium, on which a drivable area boundary line detection program is stored. When the drivable area boundary line detection program is executed by a processor, the following operations are implemented:

[0129] S1: Divide the first image according to a specified range to obtain a plurality of second images;

[0130] S2: Processing the second image using the first preset network model to obtain a line segment representation equation of the second image;

[0131] S3: sampling key points of the second image according to the line segment representation equation, connecting the key points in the second image to obtain a third image;

[0132] S4: traversing the third images that are adjacent in position, connecting the closest key points between the third images that are adjacent in position, and obtaining a boundary line of the drivable area.

[0133] Optionally, the first preset network model is trained by the following steps:

[0134] dividing the fourth image according to a specified range to obtain a plurality of fifth images;

[0135] Processing the fifth image to obtain a line segment representation equation of the fifth image;

[0136] The first preset network model is trained using the fifth image and the line segment representation equation.

[0137] Optionally, before step S2, the following steps are also included:

[0138] Processing the second image using a second preset network model to determine whether a line segment exists in the second image;

[0139] The second image with line segments is provided to the first preset network model for processing.

[0140] Optionally, the performing key point sampling on the second image according to the line segment representation equation comprises the following steps:

[0141] generating a line segment in the second image according to the line segment representation equation;

[0142] The line segment is sampled according to a specified length to obtain a key point; or, the line segment is sampled using a third preset network model to obtain a key point.

[0143] Optionally, after step S1, the following steps are further included:

[0144] Coordinates are set for each of the second images, where the coordinates are used to identify the position of the second image in the coordinate system.

[0145] Optionally, the first image is obtained by photographing a camera with a wide-angle photographing function.

[0146] Through the embodiments of the present invention, pictures taken by a fisheye camera are used as model input to obtain a larger detection range; the image block division method is used to reduce the complexity of curve fitting; and the regression method is used to fit line segments to reduce the amount of calculation of the network model.

[0147] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0148] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0149] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, controller, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0150] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for detecting the boundary line of a drivable area, It is characterized in that The method comprises the following steps: S1: Divide the first image according to a specified range to obtain a plurality of second images; S2: Processing the second image using the first preset network model to obtain a line segment representation equation of the second image; S3: sampling key points of the second image according to the line segment representation equation, connecting the key points in the second image to obtain a third image; S4: traversing the third images that are adjacent in position, connecting the closest key points between the third images that are adjacent in position, and obtaining a boundary line of a drivable area; The step of performing key point sampling on the second image according to the line segment representation equation comprises the following steps: generating a line segment in the second image according to the line segment representation equation; Sampling the line segment according to a specified length to obtain key points; or, sampling the line segment using a third preset network model to obtain key points; The first preset network model is trained by the following steps: dividing the fourth image according to a specified range to obtain a plurality of fifth images; Processing the fifth image to obtain a line segment representation equation of the fifth image includes: Processing the fifth image using Hough transform, mapping a curve or straight line having the same shape in one space to a point in another coordinate space by transforming the image space and the Hough space to vote, forming a peak value, and obtaining a line segment representation equation of the fifth image; Training a first preset network model using the fifth image and the line segment representation equation; The line segment representation equation is a curve fitting equation, a straight line equation, or a second-order polynomial.

2. The method according to claim 1, It is characterized in that Before step S2, the method further includes the following steps: Processing the second image using a second preset network model to determine whether a line segment exists in the second image; The second image with line segments is provided to the first preset network model for processing.

3. The method according to claim 1, It is characterized in that After step S1, the method further includes the following steps: Coordinates are set for each of the second images, where the coordinates are used to identify the position of the second image in the coordinate system.

4. The method according to claim 1, It is characterized in that The first image is obtained by photographing a photographing device with a wide-angle photographing function.

5. A device for detecting the boundary line of a drivable area, It is characterized in that The device comprises: An image division unit, used for dividing the first image according to a specified range to obtain a plurality of second images; an equation acquisition unit, configured to process the second image using a first preset network model to acquire a line segment representation equation of the second image; A key point sampling unit, configured to perform key point sampling on the second image according to the line segment representation equation, and connect the key points in the second image to obtain a third image; A key point connection unit, used for traversing adjacent third images, connecting the closest key points of adjacent third images, and obtaining a boundary line of a drivable area; Wherein, the key point sampling unit is also used for: generating a line segment in the second image according to the line segment representation equation; Sampling the line segment according to a specified length to obtain key points; or, sampling the line segment using a third preset network model to obtain key points; The first preset network model is trained by the following steps: dividing the fourth image according to a specified range to obtain a plurality of fifth images; Processing the fifth image to obtain a line segment representation equation of the fifth image includes: Processing the fifth image using Hough transform, mapping a curve or straight line having the same shape in one space to a point in another coordinate space by transforming the image space and the Hough space to vote, forming a peak value, and obtaining a line segment representation equation of the fifth image; Training a first preset network model using the fifth image and the line segment representation equation; The line segment representation equation is a curve fitting equation, a straight line equation, or a second-order polynomial.

6. The device according to claim 5, It is characterized in that The device also includes: The line segment judgment unit is used to process the second image using the second preset network model to determine whether there are line segments in the second image; and provide the second image with line segments to the first preset network model for processing.

7. A means of transportation, It is characterized in that The vehicle comprises: a memory, a processor and a drivable area boundary line detection program stored in the memory and executable on the processor, wherein the drivable area boundary line detection program is configured to implement the steps of the drivable area boundary line detection method according to any one of claims 1 to 4.

8. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the drivable area boundary line detection method described in any one of claims 1 to 4 are implemented.