A method and system for detecting a region of a streamer

Abstract

A method and system for detecting a flow line region, the method comprising: determining internal parameters of a camera according to Zhang's calibration method, determining external parameters of the camera through a multiple point perspective (PnP) algorithm, and calibrating a pose of the camera through the internal parameters and the external parameters; capturing a lane through the calibrated camera to obtain a lane image, and performing data fitting on the lane image to obtain a lane line fitting equation corresponding to the lane; determining two adjacent lane lines in the lane image through the lane line fitting equation, and selecting a region of interest (ROI) between the two adjacent lane lines; performing straight line identification on the ROI to obtain a straight line image of a flow line, and performing block feature extraction on the straight line image to determine a flow line block region; and determining the ROI containing the flow line block region as a flow line region between the two adjacent lane lines.

Description

Technical Field

[0001] This invention relates to the field of automotive vision, and in particular to a method and system for detecting guide line areas. Background Technology

[0002] Lane detection is an important computer vision task, commonly used in autonomous driving, driver assistance systems, and traffic monitoring. The goal of this task is to detect and identify lane lines on a road from images or videos to determine the lane a vehicle is currently in.

[0003] However, current lane detection technologies cannot accurately identify the lane a vehicle is in when the driving area includes a guide line area. This can easily provide incorrect identification information to the vehicle and driver, resulting in a poor driving experience and performance. Summary of the Invention

[0004] In view of this, it is necessary to provide a method and system for detecting guide line areas to address the aforementioned technical problems. This method and system can add guide line area detection technology to lane line detection technology, providing accurate identification information for vehicles and drivers, thereby enhancing the driving experience and performance.

[0005] One aspect of the present invention provides a method for detecting a guide line region, the method comprising:

[0006] S101, determine the camera's internal parameters according to Zhang's calibration method, determine the camera's external parameters through the multi-point perspective (PnP) algorithm, and calibrate the camera's attitude using the internal and external parameters;

[0007] S102, The calibrated camera takes pictures of the lane to obtain a lane image, and performs data fitting on the lane image to obtain a lane line fitting equation corresponding to the lane.

[0008] S103, determine two adjacent lane lines in the lane image using the lane line fitting equation, and select a region of interest (ROI) between the two adjacent lane lines.

[0009] S104, perform straight line recognition on the ROI to obtain a straight line image of the guide line, and extract block features on the straight line image to determine the block region of the guide line;

[0010] S105, the ROI containing the guide line block region is determined as the guide line region between the two adjacent lane lines.

[0011] In some embodiments, the method further includes: determining an ideal vanishing point of the guide line in the ROI; estimating the desired direction of the guide line between the two adjacent lane lines using the ideal vanishing point of the guide line; determining the slope of the guide line based on the desired direction; and identifying a straight line in the ROI using the slope.

[0012] Another aspect of the present invention provides a flow guide area detection system, the system comprising:

[0013] The camera attitude calibration module is used to determine the camera's internal parameters according to Zhang's calibration method, determine the camera's external parameters through the multi-point perspective (PnP) algorithm, and calibrate the camera's attitude using the internal and external parameters.

[0014] The lane line fitting module is used to take pictures of the lane through the calibrated camera to obtain a lane image, and to perform data fitting on the lane image to obtain a lane line fitting equation corresponding to the lane.

[0015] The ROI selection module is used to determine two adjacent lane lines in the lane image through the lane line fitting equation, and select a region of interest (ROI) between the two adjacent lane lines.

[0016] The recognition module is used to perform straight line recognition in the ROI to obtain a straight line image of the guide line, and to extract block features on the straight line image to determine the block region of the guide line;

[0017] The guide line area determination module is used to determine the ROI containing the guide line block area as the guide line area between the two adjacent lane lines.

[0018] The aforementioned guide line region detection method and system determine the camera's internal parameters using Zhang's calibration method and its external parameters using a multi-point perspective (PnP) algorithm. The camera's pose is calibrated using both internal and external parameters. The calibrated camera then captures an image of the lane, and the lane image is fitted to obtain a lane line fitting equation corresponding to the lane. Two adjacent lane lines are identified in the lane image using the lane line fitting equation, and a region of interest (ROI) is selected between these two adjacent lane lines. Straight line recognition is performed on the ROI to obtain a straight line image of the guide line, and block feature extraction is performed on this straight line image to determine the guide line block region. The ROI containing the guide line block region is defined as the guide line region between the two adjacent lane lines. This method and system, building upon lane line detection technology, adds a guide line region detection method and technology, providing accurate identification information for vehicles and drivers, thus enhancing the driving experience and performance. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of a flow guide region detection method according to an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of a guide line area detection system provided in an embodiment of the present invention;

[0022] Figure 3 This is a schematic diagram of a sharp-angled region between two lane lines in an embodiment of the present invention, used for... Figure 1 Step S105: Determine the ROI containing the guide line block region as the guide line region between the two adjacent lane lines. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description, in conjunction with the accompanying drawings and embodiments, further illustrates this application. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. Based on the description of this invention, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this invention.

[0024] Furthermore, the terms “including,” “comprising,” or any other variations thereof are intended to cover non-exclusive inclusion, which may include not only those elements listed but also other elements not expressly listed.

[0025] See Figure 1 This invention provides a method for detecting a guideline region, which can be executed by a guideline region detection system provided in this invention. The guideline region detection system can be implemented using software and / or hardware. The method includes the following steps:

[0026] Step S101: Determine the camera's internal parameters according to Zhang's calibration method, determine the camera's external parameters using the multi-point perspective (PnP) algorithm, and calibrate the camera's pose using the internal and external parameters.

[0027] Understandably, Zhang's calibration method is used to calculate camera intrinsic parameters, including information such as focal length, principal point, and distortion coefficients. This method typically requires a special calibration board with known feature points, and images of this board are taken at different angles and positions. These images are then analyzed to estimate the camera's intrinsic parameters.

[0028] The following are the steps for obtaining DVR camera intrinsic parameters using Zhang's calibration method:

[0029] Preparing the calibration board: First, you need to prepare a calibration board with known feature points. A checkerboard calibration board is commonly used because its feature points are easy to detect and track.

[0030] Capture calibration images: Mount a camera on the vehicle and capture images of the calibration plate from different angles and positions. Ensure coverage of as many different viewpoints and distances as possible to improve calibration accuracy.

[0031] Feature point detection: Use computer vision software or libraries (such as OpenCV) to detect feature points in the calibration board image. These feature points are usually intersections or corners on the calibration board.

[0032] Establishing the correspondence between image coordinates and physical coordinates: For each captured image of the calibration board, it is necessary to establish the correspondence between image coordinates (pixel coordinates) and physical coordinates (actual coordinates on the calibration board). This is accomplished by measuring the actual dimensions of feature points on the calibration board.

[0033] Perform the calibration algorithm: calculate the camera's intrinsic parameters using the aforementioned correspondence. This will include information such as focal length, principal point coordinates, and distortion coefficients.

[0034] Understandably, the PnP algorithm, short for "Perspective-n-Point" algorithm, is a technique in the fields of computer vision and machine vision used to solve the problem of camera pose estimation. Specifically, the PnP algorithm is used to estimate the camera's pose (position and orientation) based on known points in 3D space and their projected positions in an image.

[0035] The input to the PnP algorithm typically includes the following information:

[0036] A set of known points in three-dimensional space is usually represented as P(X,Y,Z).

[0037] The corresponding projection points of these 3D points in the camera image are usually represented as p(u, v).

[0038] The camera's intrinsic parameter matrix M includes focal length, principal point position, and distortion parameters. This step is achieved using the Zhang calibration method described above.

[0039] The camera's pose can be obtained using the PnP algorithm, typically given as a rotation matrix (R) and a translation matrix (t), describing the camera's position and orientation. The camera's position and orientation indicate the vehicle's position and orientation.

[0040] Step S102: Take a picture of the lane using the calibrated camera to obtain a lane image, and perform data fitting on the lane image to obtain a lane line fitting equation corresponding to the lane.

[0041] In one embodiment, a lane image is obtained by capturing a picture of the lane using a calibrated camera. The lane image is then fitted using the least squares method. Fitting lane lines using the least squares method is a technique in computer vision and autonomous driving fields used to extract the shape of lane lines from images captured by a camera. This method approximates the shape of the lane lines by fitting a mathematical model, typically a polynomial curve. Fitting the lane image using the least squares method first uses image processing techniques, such as edge detection and color thresholding, to detect lane lines in the lane image. This generates a binary mask image of the lane lines, where lane line pixels are white and other areas are black. Then, the pixel coordinates of the white pixels of the lane lines are extracted from the lane line mask image. These coordinates are used to fit the mathematical model of the lane lines. The fitted mathematical model is:

[0042] a i x+b i y+c i =0; i = 0, ..., n-1; i represents the index of the lane line.

[0043] Then, using the lane line model, a representation of the lane lines is drawn on the original image to visualize it.

[0044] Step S103: Determine two adjacent lane lines in the lane image using the lane line fitting equation, and select a region of interest (ROI) between the two adjacent lane lines.

[0045] In one embodiment, two adjacent lane lines are determined in the lane image using the lane line fitting equation. First, based on the projection relationship, the detected pixel coordinates (u... ij ,v ij Convert to 3D world coordinates (x) ij ,y ij ,z ij1). This can be achieved using the inverse of the camera intrinsic parameter matrix M and the camera pose matrix (Rc,t), as well as the perspective projection relationship, as shown in the following formula:

[0046]

[0047] i = 0, ..., n-1, where i is the index number of the lane line.

[0048] j = 0, ..., k-1, where k is a point on each lane line.

[0049] Then, for each lane line, calculate all points (x) ij ,y ij ,z ij The mean y-coordinate of lane 1 is calculated. Then, the differences between the mean y-coordinates of different lane lines are examined to determine if they are close. This allows the identification of two adjacent lane lines in the lane image.

[0050] Understandably, ROI (Region of Interest) is a commonly used concept in image or video processing, referring to a region in an image that is defined as important or of interest. ROI is typically a subset of the entire image selected and extracted based on specific conditions or task requirements. In this invention, the ROI is the region between two adjacent lane lines.

[0051] Step S104: Perform straight line recognition on the ROI to obtain a straight line image of the guide line, and extract block features on the straight line image to determine the block region of the guide line.

[0052] Before performing line recognition in the Region of Interest (ROI), the vanishing point of the ideal guide line is determined within the ROI. The desired direction of the guide line between two adjacent lane lines is estimated using this vanishing point. The slope of the guide line is determined based on this desired direction, and line recognition is performed on the ROI based on this slope. Classic line extraction algorithms, such as Hough transform, LSD (Line Segment Detector), FLD (Fast Line Detector), EDLines (Adaptive Edge Line Detector), and LSM (Least Squares Line Fitting), are used to extract lines within the ROI. Based on the slope of the guide line, lines parallel to the lane line direction are removed. The remaining lines are then used for line recognition to obtain the guide line line image.

[0053] Then, by extracting block features from the straight line image, the ratio of the area of ​​each block feature to the area of ​​the smallest enclosing rectangle is obtained. The smallest enclosing rectangle is the smallest rectangle that surrounds the entire block feature. When the ratio is less than a preset threshold, the region is removed, and the remaining region is the guide line block region.

[0054] Step S105: The ROI containing the guide line block area is determined as the guide line area between the two adjacent lane lines.

[0055] Understandably, if the ROI region includes the guide line block region, it is considered a guide line region. However, in the sharp corner portion between two lane lines, such as... Figure 3 As shown, the pointed part marked by circle A is actually part of the guide line area. However, in practice, because this part is relatively narrow and does not include blocky areas, the blocky area of ​​the guide line cannot be detected and identified. Therefore, in some implementations, it is necessary to calculate all points (x) of each lane line calculated in step S103. ij ,y ij ,z ij The mean of the y-coordinate of ,1) is used to determine the area. Because the width of this area is small, the mean of the y-coordinate of the corresponding lane lines is lower than a certain threshold. Therefore, when the difference between the mean of the y-coordinates of different lane lines is less than a certain threshold w, these areas are also judged as guide line areas.

[0056] The aforementioned guideline region detection method determines the camera's internal parameters using Zhang's calibration method and its external parameters using a multi-point perspective (PnP) algorithm. The camera's pose is then calibrated using both internal and external parameters. The calibrated camera captures an image of the lane, and this image is fitted to obtain a lane line fitting equation. Two adjacent lane lines are identified in the lane image using the lane line fitting equation, and a region of interest (ROI) is selected between these two adjacent lane lines. Straight line recognition is performed within the ROI to obtain a straight line image of the guideline, and block feature extraction is performed on this image to determine the guideline block region. The ROI containing the guideline block region is defined as the guideline region between the two adjacent lane lines. This method adds a guideline region detection method and technology to lane line detection, providing accurate identification information for vehicles and drivers, thus enhancing the driving experience and performance.

[0057] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0058] In one embodiment, such as Figure 2 As shown, a schematic diagram of a guideline region detection system is provided, including: a camera attitude calibration module 210, a lane line fitting module 220, an ROI selection module 230, a recognition module 240, and a guideline region determination module 250; wherein:

[0059] The camera attitude calibration module 210 is used to determine the camera's internal parameters according to Zhang's calibration method, determine the camera's external parameters through the multi-point perspective (PnP) algorithm, and calibrate the camera's attitude using the internal and external parameters.

[0060] The lane line fitting module 220 is used to take pictures of the lane through the calibrated camera to obtain a lane image, and to perform data fitting on the lane image to obtain a lane line fitting equation corresponding to the lane.

[0061] The ROI selection module 230 is used to determine two adjacent lane lines in the lane image through the lane line fitting equation, and select a region of interest (ROI) between the two adjacent lane lines.

[0062] The recognition module 240 is used to perform straight line recognition in the ROI to obtain a straight line image of the guide line, and to extract block features on the straight line image to determine the block region of the guide line.

[0063] The guide line region determination module 250 is used to determine the ROI containing the guide line block region as the guide line region between the two adjacent lane lines.

[0064] In one embodiment, the vehicle driving mode setting system further includes:

[0065] Before the identification module 240, an ideal guide vanishing point is determined in the ROI. The desired direction of the guide line between the two adjacent lane lines is estimated by the ideal guide vanishing point. The slope of the guide line is determined according to the desired direction. The ROI is then identified as a straight line based on the slope.

[0066] Specific limitations regarding the flow guide area detection system can be found in the limitations of the flow guide area detection method described above, and will not be repeated here. Each module in the aforementioned flow guide area detection system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0067] The guide line region detection system of this embodiment calibrates the camera's orientation using a camera calibration module; it then captures a lane image using the calibrated camera, and fits this image to obtain a lane line fitting equation corresponding to the lane; it identifies two adjacent lane lines in the lane image using the lane line fitting equation, and selects a region of interest (ROI) between these two adjacent lane lines using an ROI selection module; it performs straight line recognition on the ROI using a recognition module to obtain a straight line image of the guide line, and extracts block features from this image to determine the guide line block region; finally, the guide line region determination module identifies the ROI containing the guide line block region as the guide line region between the two adjacent lane lines. This system adds guide line region detection technology to lane line detection technology, providing accurate identification information for vehicles and drivers, thus improving the driving experience and performance.

[0068] In one embodiment, a vehicle is provided that includes the aforementioned guide vane area detection system. Specific limitations regarding the vehicle can be found in the above description of the guide vane area detection system, and will not be repeated here.

[0069] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements any one of the steps in the above-described guide line region detection method.

[0070] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0071] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0072] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for detecting guide line areas, used to detect guide line areas in a lane, characterized in that, Includes the following steps: S101, determine the camera's internal parameters according to Zhang's calibration method, determine the camera's external parameters through the multi-point perspective (PnP) algorithm, and calibrate the camera's attitude using the internal and external parameters; S102, The calibrated camera takes pictures of the lane to obtain a lane image, and performs data fitting on the lane image to obtain a lane line fitting equation corresponding to the lane. S103, determine two adjacent lane lines in the lane image using the lane line fitting equation, and select a region of interest (ROI) between the two adjacent lane lines. S104, perform straight line recognition on the ROI to obtain a straight line image of the guide line, and extract block features on the straight line image to determine the block region of the guide line; S105, the ROI containing the guide line block region is determined as the guide line region between the two adjacent lane lines; Prior to step S104, the method further includes: In the ROI, the ideal vanishing point of the guide line is determined. The desired direction of the guide line between the two adjacent lane lines is estimated based on the ideal vanishing point of the guide line. The slope of the guide line is determined based on the desired direction. The ROI is then used to identify straight lines based on the slope. The step of determining the slope of the guide line based on the desired direction, and then using the slope to identify the ROI as a straight line, includes: Based on the slope of the guide line, straight lines parallel to the lane line direction are removed, and the remaining part is used for straight line recognition to obtain a straight line image of the guide line. Step S104 involves extracting block features from the straight line image to determine the block region of the guide line, including: By extracting block features from the straight line image, the ratio of the area of ​​each block feature to the area of ​​the smallest enclosing rectangle is obtained. The smallest enclosing rectangle is the smallest rectangle that encloses the entire block feature. When the ratio is less than a preset threshold, the region is removed, and the remaining region is the guide line block region.

2. A flow guide area detection system, characterized in that, It includes a camera attitude calibration module, a lane line fitting module, an ROI selection module, a recognition module, and a guide line region determination module; among which: The camera attitude calibration module is used to determine the camera's internal parameters according to Zhang's calibration method, determine the camera's external parameters through the multi-point perspective (PnP) algorithm, and calibrate the camera's attitude using the internal and external parameters. The lane line fitting module is used to take pictures of the lane through the calibrated camera to obtain a lane image, and to perform data fitting on the lane image to obtain a lane line fitting equation corresponding to the lane. The ROI selection module is used to determine two adjacent lane lines in the lane image using the lane line fitting equation, and to select a region of interest (ROI) between the two adjacent lane lines. The recognition module is used to perform straight line recognition in the ROI to obtain a straight line image of the guide line, and to extract block features on the straight line image to determine the block region of the guide line; The guide line region determination module is used to determine the ROI containing the guide line block region as the guide line region between the two adjacent lane lines. The system also includes: Before the identification module, the ideal vanishing point of the guide line is determined in the ROI. The desired direction of the guide line between the two adjacent lane lines is estimated by the ideal vanishing point of the guide line. The slope of the guide line is determined according to the desired direction. The ROI is then identified as a straight line based on the slope. The identification module includes: Based on the slope of the guide line, straight lines parallel to the lane line direction are removed, and the remaining part is used for straight line recognition to obtain a straight line image of the guide line. The identification module further includes: By extracting block features from the straight line image, the ratio of the area of ​​each block feature to the area of ​​the smallest enclosing rectangle is obtained. The smallest enclosing rectangle is the smallest rectangle that encloses the entire block feature. When the ratio is less than a preset threshold, the region is removed, and the remaining region is the guide line block region.

3. A car, characterized in that, The vehicle includes the guide vane area detection system as described in claim 2.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method of claim 1.

Images