Diversion line region detection method and system

Abstract

A diversion line region detection method and system. The method comprises: determining internal parameters of a camera on the basis of a Zhang's calibration method, determining external parameters of the camera by means of a perspective-n-point (PnP) algorithm, and calibrating the attitude of the camera by means of the internal parameters and the external parameters; photographing a lane by means of the calibrated camera to obtain a lane image, and performing data fitting on the lane image to obtain a lane line fitting equation corresponding to the lane; determining two adjacent lane lines in the lane image by means of the lane line fitting equation, and selecting a region of interest (ROI) between the two adjacent lane lines; performing straight line recognition in the ROI to obtain a straight line image of a diversion line, and performing block feature extraction on the straight line image to determine a diversion line block region; and determining the ROI containing the diversion line block region as a diversion line region between the two adjacent lane lines.

Description

Guide line area detection method and system

[0001] This application claims priority to Chinese patent application No. 202311514991.0 filed on November 15, 2023, entitled “A guide line area detection method and system,” the entire contents of which are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of display technology, and in particular to a display panel and a display device having the display panel. Background Art

[0003] Lane detection is an important computer vision task, commonly used in autonomous driving, driver assistance systems, and traffic monitoring. The goal of this task is to detect and identify lane lines on the road from images or videos to determine the current lane of the vehicle. Technical issues

[0004] The current existing lane line detection technology cannot accurately identify the lane where the vehicle is traveling when the vehicle's driving area includes a guide line area, which can easily provide incorrect identification information to the vehicle and driver, resulting in a poor driving experience and performance. Technical Solutions

[0005] In view of this, it is necessary to provide a guide line area detection method and system to address the above technical issues. This method and system can be added to the lane line detection technology to provide accurate identification information to the vehicle and driver, improving the driving experience and performance.

[0006] One aspect of the present invention provides a method for detecting a guide line area, the method comprising:

[0007] S101, determining the intrinsic parameters of a camera according to Zhang's calibration method, determining the extrinsic parameters of the camera using a multi-point perspective (PnP) algorithm, and calibrating the posture of the camera using the intrinsic and extrinsic parameters;

[0008] S102, photographing the lane with the calibrated camera to obtain a lane image, and performing data fitting on the lane image to obtain a lane line fitting equation corresponding to the lane;

[0009] S103, determining two adjacent lane lines in the lane image using the lane line fitting equation, and selecting a region of interest (ROI) between the two adjacent lane lines;

[0010] S104, performing straight line recognition in the ROI to obtain a straight line image of the guide line, and performing block feature extraction on the straight line image to determine a block area of ​​the guide line;

[0011] S105: Determine the ROI including the guide line block area as the guide line area between the two adjacent lane lines.

[0012] In some embodiments, the method further includes: determining an ideal guide line vanishing point in the ROI, estimating the desired direction of the guide line between the two adjacent lane lines through the ideal guide line vanishing point, determining the slope of the guide line according to the desired direction, and identifying a straight line of the ROI through the slope.

[0013] Another aspect of the present invention provides a guide wire area detection system, the system comprising:

[0014] A camera attitude calibration module, configured to determine the internal parameters of the camera according to the Zhang calibration method, determine the external parameters of the camera using a multi-point perspective (PnP) algorithm, and calibrate the attitude of the camera using the internal and external parameters;

[0015] a lane line fitting module, configured to photograph a lane using the calibrated camera to obtain a lane image, and perform data fitting on the lane image to obtain a lane line fitting equation corresponding to the lane;

[0016] a ROI selection module, configured to determine two adjacent lane lines in the lane image using the lane line fitting equation, and select a region of interest (ROI) between the two adjacent lane lines;

[0017] An identification module is used to perform straight line identification in the ROI to obtain a straight line image of the guide line, and perform block feature extraction on the straight line image to determine the block area of ​​the guide line;

[0018] The guide line area determination module is used to determine the ROI containing the guide line block area as the guide line area between the two adjacent lane lines. Beneficial effects

[0019] The above-mentioned guide line area detection method and system determines the camera's internal parameters using the Zhang calibration method, determines the camera's external parameters using a multi-point perspective (PnP) algorithm, and calibrates the camera's pose using these internal and external parameters. The calibrated camera then captures the lane to obtain a lane image, performs data fitting on the lane image, and obtains a lane line fitting equation corresponding to the lane. Using the lane line fitting equation, two adjacent lane lines are identified in the lane image and a region of interest (ROI) is selected between the two adjacent lane lines. Lines are identified in the ROI to obtain a straight line image of the guide lines, and block features are extracted on the straight line image to determine the guide line block area. The ROI containing the guide line block area is then determined as the guide line area between the two adjacent lane lines. This method and system, in addition to lane line detection technology, adds a guide line area detection method and technology, providing accurate identification information to vehicles and drivers, enhancing the driving experience and performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0021] FIG1 is a flow chart of a guide line area detection method according to an embodiment of the present invention;

[0022] FIG2 is a schematic diagram of a guide line area detection system provided by an embodiment of the present invention;

[0023] FIG3 is a schematic diagram of a sharp-angle region between two lane lines in an embodiment of the present invention, which is used in step S105 of FIG1 : determining the ROI including the guide line block region as the guide line region between the two adjacent lane lines. Modes for Carrying Out the Invention

[0024] In order to make the purpose, technical solutions and advantages of this application more clear, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application. Based on the description of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0025] In addition, the terms "comprises," "comprising," or any other variations thereof, are intended to cover a non-exclusive inclusion that may include elements not specifically listed in addition to the listed elements.

[0026] Referring to FIG1 , a guide line area detection method provided in an embodiment of the present invention is shown. The guide line area detection method can be performed by a guide line area detection system provided in an embodiment of the present invention. The guide line area detection system can be implemented in software and / or hardware. The guide line area detection method includes the following steps:

[0027] Step S101: determining the internal parameters of a camera according to Zhang's calibration method, determining the external parameters of the camera by using a multi-point perspective (PnP) algorithm, and calibrating the posture of the camera by using the internal and external parameters.

[0028] Zhang calibration is a method for calculating camera intrinsic parameters, including information such as focal length, principal point, and distortion coefficients. This method typically requires using a special calibration plate with known feature points. Images of this plate are then captured at various angles and positions. The camera's intrinsic parameters are then estimated by analyzing these images.

[0029] The following are the steps to obtain the intrinsic parameters of the DVR camera using Zhang's calibration method:

[0030] Prepare a calibration plate: First, you need to prepare a calibration plate with known feature points. A checkerboard calibration plate is usually used because its feature points are easy to detect and track.

[0031] Capture calibration images: Mount the camera on the vehicle and capture images of the calibration plate at different angles and positions. Make sure to cover as many different view angles and distances as possible to improve calibration accuracy.

[0032] Detect feature points: Use computer vision software or libraries (such as OpenCV) to detect feature points in the calibration plate image. These feature points are usually intersections or corners on the calibration plate.

[0033] Establishing the correspondence between image coordinates and physical coordinates: For each captured calibration plate image, it is necessary to establish the correspondence between the image coordinates (pixel coordinates) and the physical coordinates (actual coordinates on the calibration plate). This is done by measuring the actual size of the feature points on the calibration plate.

[0034] Execute the calibration algorithm: Calculate the camera's intrinsic parameters using the correspondence relationship. This includes information such as focal length, principal point coordinates, and distortion coefficients.

[0035] The Perspective-n-Point (PnP) algorithm, short for "Perspective-n-Point," is a technique in computer vision and machine vision that addresses the problem of camera pose estimation. Specifically, the PnP algorithm estimates the camera's pose (position and orientation) based on known 3D points and their projections in an image.

[0036] The input to the PnP algorithm typically includes the following information:

[0037] A set of known points in three-dimensional space, usually denoted as P(X, Y, Z).

[0038] The corresponding projection points of these 3D points in the camera image are usually denoted as p(u, v).

[0039] The camera's intrinsic parameter matrix M, including focal length, principal point position, and distortion parameters, is obtained using the Zhang calibration method.

[0040] The PnP algorithm can be used to obtain the camera's pose, which is usually given in the form of a rotation matrix (R) and a translation matrix (t), describing the camera's position and orientation. The camera's position and orientation indicate the car's position and orientation.

[0041] Step S102: photographing the lane with the calibrated camera to obtain a lane image, and performing data fitting on the lane image to obtain a lane line fitting equation corresponding to the lane.

[0042] In one embodiment, the lane is photographed by the calibrated camera to obtain a lane image, and the lane image is data fitted by the least squares method. Using the least squares method to fit the lane line is a technology in the field of computer vision and autonomous driving, which is used to extract the shape of the lane line from the image captured by the camera. This method approximates the shape of the lane line by fitting a mathematical model, usually a polynomial curve. Using the least squares method to fit the lane image data first uses image processing techniques such as edge detection, color threshold segmentation, etc. to detect the lane lines in the lane image. This will generate a binary mask image of the lane line, in which the lane line pixels are white and other areas are black. The pixel coordinates of the white pixels of the lane line are then extracted from the lane line mask image. These coordinates will be used to fit the mathematical model of the lane line. The fitted mathematical model is:

[0043] a i x+b i y+c i =0;i=0,…,n-1; i represents the index of the lane line.

[0044] The lane line model is then used to draw a representation of the lane lines on the original image to visualize them.

[0045] Step S103: determining two adjacent lane lines in the lane image using the lane line fitting equation, and selecting a region of interest (ROI) between the two adjacent lane lines.

[0046] In one embodiment, two adjacent lane lines are determined in the lane image by the lane line fitting equation. First, the pixel coordinates (u ij ,v ij ) is converted to 3D world coordinates (x ij ,y ij ,z ij ,1). This can be achieved by the inverse of the camera intrinsic parameter matrix M and the camera pose matrix (Rc, t) and the perspective projection relationship, as follows:

[0047]

[0048] i=0,⋯,n-1, is the lane line index.

[0049] j=0,⋯,k-1, where k is a point on each lane line.

[0050] Then for each lane line, calculate all points (x ij ,y ij ,z ij ,1). Then, the difference between the y-coordinate means of different lane lines is checked to determine whether they are close. In this way, two adjacent lane lines can be determined in the lane image.

[0051] As you can understand, ROI (Region of Interest) is a commonly used concept in image or video processing. It refers to an area of ​​an image that is defined as important or interesting. An ROI is typically a region of the entire image that is selected and extracted based on specific conditions or task requirements. In this case, the ROI is the area between two adjacent lane lines.

[0052] Step S104: performing straight line recognition in the ROI to obtain a straight line image of the guide line, and performing block feature extraction on the straight line image to determine a block area of ​​the guide line.

[0053] Before performing line recognition within the ROI, the ideal vanishing point of the guide line is determined within the ROI. This ideal vanishing point is used to estimate the desired direction of the guide line between two adjacent lane lines. The slope of the guide line is then determined based on the desired direction, and line recognition is performed within the ROI based on this slope. Classic line extraction algorithms, such as Hough transform, LSD (line segment detector), FLD (fast line detector), EDLines (adaptive edge line detector), and LSM (least squares line fitting), are used to extract lines within the region of interest (ROI). Lines parallel to the lane lines are removed based on the slope of the guide line. Line recognition is then performed on the remaining lines to obtain a guide line image.

[0054] Then, block features are extracted from the straight line image to obtain the ratio of the area of ​​each block feature to the area of ​​the minimum outer enclosing rectangle. The minimum outer enclosing rectangle is the smallest rectangle that surrounds the entire block feature. When the ratio is less than a preset threshold, the area is eliminated, and the remaining area is the guide line block area.

[0055] Step S105: determining the ROI including the guide line block area as the guide line area between the two adjacent lane lines.

[0056] It is understandable that if the ROI area includes the guide line block area, it is the guide line area. However, the sharp corner between two lane lines, as shown in Figure 3, the sharp corner marked by circle A, is also a guide line area in practice. However, in practice, since the width of this part is relatively narrow and this part does not include the block area, the guide line block area cannot detect and identify this part. Therefore, in some embodiments, it is necessary to calculate for each lane line in step S103, all points (x ij ,y ij ,z ij ,1) The mean value of the y coordinate is judged. Because the width of this area is small, the mean value of the y coordinates of different lane lines is lower than a certain threshold. Therefore, when the difference between the mean values ​​of the y coordinates of different lane lines is less than a certain threshold w, these areas are also judged as guide line areas.

[0057] The above-mentioned guide line area detection method determines the camera's internal parameters using the Zhang calibration method, determines the camera's external parameters using a multi-point perspective (PnP) algorithm, and calibrates the camera's pose using these internal and external parameters. The calibrated camera then captures the lane to obtain a lane image, performs data fitting on the lane image, and obtains a lane line fitting equation corresponding to the lane. Using the lane line fitting equation, two adjacent lane lines are identified in the lane image, and a region of interest (ROI) is selected between the two adjacent lane lines. Lines are identified in the ROI to obtain a straight line image of the guide lines, and block features are extracted on the straight line image to determine the guide line block area. The ROI containing the guide line block area is then determined as the guide line area between the two adjacent lane lines. This method adds a guide line area detection method and technology to lane line detection techniques, providing accurate identification information to vehicles and drivers, enhancing the driving experience and performance.

[0058] It should be understood that, although the various steps in the flowchart of FIG1 are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in FIG1 may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these sub-steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0059] In one embodiment, as shown in FIG2 , a schematic diagram of a guide line area detection system is provided, including: a camera posture calibration module 210 , a lane line fitting module 220 , a ROI selection module 230 , a recognition module 240 , and a guide line area determination module 250 ; wherein:

[0060] The camera attitude calibration module 210 is configured to determine the internal parameters of the camera according to the Zhang calibration method, determine the external parameters of the camera using a multi-point perspective (PnP) algorithm, and calibrate the attitude of the camera using the internal and external parameters;

[0061] The lane line fitting module 220 is configured to capture a lane using the calibrated camera to obtain a lane image, and perform data fitting on the lane image to obtain a lane line fitting equation corresponding to the lane;

[0062] The ROI selection module 230 is configured to determine two adjacent lane lines in the lane image using the lane line fitting equation, and select a region of interest (ROI) between the two adjacent lane lines;

[0063] The recognition module 240 is used to perform straight line recognition in the ROI to obtain a straight line image of the guide line, and perform block feature extraction on the straight line image to determine the block area of ​​the guide line;

[0064] The guide line area determination module 250 is configured to determine the ROI including the guide line block area as the guide line area between the two adjacent lane lines.

[0065] In one embodiment, the vehicle driving mode setting system further includes:

[0066] Before the recognition module 240, an ideal guide line vanishing point is determined in the ROI, the desired direction of the guide line between the two adjacent lane lines is estimated based on the ideal guide line vanishing point, the slope of the guide line is determined according to the desired direction, and the ROI is identified as a straight line based on the slope.

[0067] The specific definitions of the guide line area detection system can be found in the definitions of the guide line area detection method above and will not be repeated here. Each module in the aforementioned guide line area detection system can be implemented in whole or in part through software, hardware, or a combination thereof. Each of these modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0068] The guide line area detection system of this embodiment calibrates the camera's posture using a camera calibration module. The calibrated camera captures a lane to obtain a lane image, and the lane line fitting module performs data fitting on the lane image to obtain a lane line fitting equation corresponding to the lane. The lane line fitting equation is used to identify two adjacent lane lines in the lane image, and a region of interest (ROI) is selected between the two adjacent lane lines using a ROI selection module. A recognition module identifies lines within the ROI to obtain a straight line image of the guide lines, and performs block feature extraction on the straight line image to determine the guide line block area. Finally, the guide line area determination module identifies the ROI containing the guide line block area as the guide line area between the two adjacent lane lines. This system integrates lane line detection technology with guide line area detection, providing accurate identification information to the vehicle and driver, enhancing the driving experience and performance.

[0069] In one embodiment, a car is provided, comprising the guide line area detection system described above. Specific definitions of the car can be found in the above definition of the guide line area detection system, which will not be repeated here.

[0070] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, any step of the above-mentioned guide line area detection method is implemented.

[0071] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0072] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0073] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A guide line area detection method for detecting a guide line area in a lane, It is characterized in that The steps include: S101, determining internal parameters of a camera according to Zhang's calibration method, determining external parameters of the camera by a multi-point perspective (PnP) algorithm, and calibrating the posture of the camera by the internal parameters and the external parameters; S102, photographing the lane with the calibrated camera to obtain a lane image, and performing data fitting on the lane image to obtain a lane line fitting equation corresponding to the lane; S103, determining two adjacent lane lines in the lane image by using the lane line fitting equation, and selecting a region of interest (ROI) between the two adjacent lane lines; S104, performing straight line recognition in the ROI to obtain a straight line image of the guide line, and performing block feature extraction on the straight line image to determine a block area of ​​the guide line; S105: Determine the ROI including the guide line block area as the guide line area between the two adjacent lane lines.

2. The method according to claim 1, It is characterized in that Before step S4, the method further includes: An ideal guide line vanishing point is determined in the ROI, an expected direction of the guide line between the two adjacent lane lines is estimated through the ideal guide line vanishing point, a slope of the guide line is determined according to the expected direction, and a straight line of the ROI is identified through the slope.

3. The method according to claim 2, It is characterized in that Determining the slope of the guide line according to the expected direction, and identifying the ROI by the slope includes: According to the slope of the guide line, the straight lines parallel to the lane line direction are eliminated, and the remaining parts are subjected to straight line recognition to obtain a guide line straight line image.

4. The method according to claim 1, It is characterized in that The step S4 performs block feature extraction on the straight line image to determine the block area of ​​the guide line, including: By performing block feature extraction on the straight line image, the ratio of the area of ​​each block feature to the area of ​​the minimum outer enclosing rectangle is obtained. The minimum outer enclosing rectangle is the smallest rectangle that surrounds the entire block feature. When the ratio is less than a preset threshold, the area is eliminated, and the remaining area is the guide line block area.

5. A guide line area detection system, It is characterized in that It includes camera attitude calibration module, lane line fitting module, ROI selection module, recognition module, and guide line area determination module; among which: The camera attitude calibration module is used to determine the internal parameters of the camera according to Zhang's calibration method, determine the external parameters of the camera through a multi-point perspective (PnP) algorithm, and calibrate the attitude of the camera through the internal parameters and the external parameters; The lane line fitting module is used to photograph the lane with the calibrated camera to obtain a lane image, and perform data fitting on the lane image to obtain a lane line fitting equation corresponding to the lane; The ROI selection module is used to determine two adjacent lane lines in the lane image by using the lane line fitting equation, and select a region of interest (ROI) between the two adjacent lane lines; The recognition module is used to perform straight line recognition in the ROI to obtain a straight line image of the guide line, and perform block feature extraction on the straight line image to determine the block area of ​​the guide line; The guide line area determination module is used to determine the ROI containing the guide line block area as the guide line area between the two adjacent lane lines.

6. The system according to claim 5, It is characterized in that The system further comprises: Before the recognition module, an ideal guide line vanishing point is determined in the ROI, the desired direction of the guide line between the two adjacent lane lines is estimated through the ideal guide line vanishing point, the slope of the guide line is determined according to the desired direction, and the ROI is identified as a straight line through the slope.

7. The system according to claim 5, It is characterized in that The identification module comprises: According to the slope of the guide line, the straight lines parallel to the lane line direction are eliminated, and the remaining parts are subjected to straight line recognition to obtain a guide line straight line image.

8. The system according to claim 5, It is characterized in that The identification module also includes: By performing block feature extraction on the straight line image, the ratio of the area of ​​each block feature to the area of ​​the minimum outer enclosing rectangle is obtained. The minimum outer enclosing rectangle is the smallest rectangle that surrounds the entire block feature. When the ratio is less than a preset threshold, the area is eliminated, and the remaining area is the guide line block area.

9. A car, It is characterized in that The automobile comprises the guide line area detection system according to any one of claims 5 to 8.

10. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images