A method and system for automatically extracting lane lines based on regions of interest

Abstract

The present invention discloses a method and system for automatically extracting lane lines based on regions of interest, including: light input control: collecting images, adjusting adjustment parameters according to the illumination scene of the images and the mapping relationship obtained by pre-illumination suitability training; foreign object detection and processing: selecting regions of interest from images, establishing energy functions and performing cyclic iterations, solving the pixel combination of foreign objects, setting the pixels belonging to foreign objects to zero, and obtaining the region of interest after the foreign objects are removed; lane line extraction: grayscale reading of the region of interest after the foreign objects are removed, binary classification according to grayscale similarity, setting loss function convergence and termination conditions, stopping iteration after the termination conditions are met, completing the pixel distinction between lane lines and non-lane lines, and finally extracting lane lines. The present invention controls the amount of light input and detects foreign objects in the lane line area to remove the corresponding pixels, thereby achieving both fast recognition reaction speed and accurate recognition results.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving environment perception technology, and in particular to a method and system for automatically extracting lane lines based on a region of interest. Background Art

[0002] The autonomous driving system is divided into: perception system and decision system. Among them, the perception system is composed of vehicle positioning, moving obstacle detection, traffic sign detection and lane line extraction. The lane line extraction using visual sensors is mainly divided into deep learning methods and traditional image processing methods, but both have obvious shortcomings. In the process of using deep learning methods to extract lane lines, a large amount of prior information needs to be collected. Moreover, deep learning methods generally have high time complexity and consume more storage space, which is contrary to the requirements of fast response and low latency of autonomous driving systems. Although traditional methods do not require a large amount of prior information and a large storage space, when extracting lane lines, if the ambient lighting changes, the threshold needs to be manually modified, otherwise the adaptive extraction of lane lines cannot be completed. Moreover, when there are interference from other vehicles or pedestrians in the lane, the integrity of the extracted lane lines cannot be guaranteed.

[0003] In nature, many animals will shrink their pupils to reduce the amount of light entering when the ambient light intensity is high. When the light intensity is weak, the pupils will be enlarged to maximize the amount of light entering, so as to improve their ability to adapt to different lighting scenarios. In addition, the human visual system can "pay attention" to the image currently captured by the eyes, actively ignoring the background that is not related to the target, so as to focus on the target object. If the adjustment of the pupil size of animals and the human visual attention mechanism are applied to the process of lane line extraction by autonomous vehicles, the lane line extraction of autonomous vehicles can be made more accurate, and the safety of unmanned vehicles can be better guaranteed while meeting the fast response and low latency requirements of the autonomous driving system. Summary of the invention

[0004] In view of the obvious shortcomings of the two lane line extraction methods in the prior art and the problem that they cannot adapt well to the application scenarios of autonomous driving, the present invention provides a method and system for automatic lane line extraction based on a region of interest. By extracting the region of interest, the parameters of the grayscale nonlinear transformation under different lighting scenes are trained, the "incoming light" of the autonomous driving vehicle under different lighting conditions is controlled, and foreign objects within the lane line range are detected and removed to avoid affecting the lane line extraction. Then, each pixel is classified to accurately distinguish between pixels belonging to a class of lane lines and pixels that are not lane lines.

[0005] The following is the technical solution of the present invention.

[0006] A method for automatically extracting lane lines based on a region of interest comprises the following steps:

[0007] Light input control: collect images and adjust the adjustment parameters according to the image's lighting scene and the mapping relationship obtained through pre-training on lighting suitability;

[0008] Foreign body detection and processing: select the region of interest from the image, establish an energy function and perform loop iteration to solve the pixel combination of the foreign body, set the pixels belonging to the foreign body to zero, and obtain the region of interest after removing the foreign body;

[0009] Lane line extraction: After removing foreign objects, the region of interest is read in grayscale, and binary classification is performed according to grayscale similarity. The loss function convergence and termination conditions are set. After the termination conditions are met, the iteration is stopped, and the pixel distinction between lane lines and non-lane lines is completed, and finally the lane lines are extracted.

[0010] The present invention improves the detection accuracy of the area of ​​interest by controlling the amount of incoming light, detects foreign objects in the lane line area and clears the corresponding pixels, so as to avoid the negative impact of the appearance of foreign objects on lane line extraction during autonomous driving. Since the control of the amount of incoming light relies on the pre-conducted lighting suitability training, it has the effects of fast recognition reaction speed and accurate recognition results.

[0011] Preferably, the process of lighting suitability training includes: setting the image size acquired by the visual sensor to W×H, the grayscale range to [0, L-1], obtaining the traffic sign bounding box under different lighting scenes, and selecting the pixels within the bounding box as the region of interest, selecting adjustment parameters under different lighting scenes to extract lane lines, and establishing a mapping relationship between the adjustment parameters and the lighting scene accordingly.

[0012] Preferably, the step of selecting adjustment parameters under different lighting scenes to extract lane lines and accordingly establishing a mapping relationship between the adjustment parameters and the lighting scenes includes:

[0013] Calculate the grayscale distribution of each similar region of interest under various lighting environments:

[0014] p(s k ) = g k ;

[0015] Among them, s k Indicates that a pixel belongs to the kth gray level, g k Represents the total number of pixels belonging to gray level k in the image. The acquired RGB image is transformed nonlinearly to complete the mapping from gray level to high gray level or from high gray level to low gray level. Its form is:

[0016] p′=cp α ;

[0017] Where p is the input grayscale of the current image, p′ is the output grayscale of the current image, c is the grayscale scaling ratio, and α is the grayscale nonlinear transformation factor. k , select different c and α, extract the complete lane line, and establish g k The mapping f(g k )→c,α, completing the illumination adaptation training.

[0018] Preferably, the process of controlling the amount of light entering includes:

[0019] Extract the region of interest from the image and calculate the histogram distribution g k , using the trained mapping relationship f(g k )→c,α, select the c and α parameters suitable for the current lighting scene to complete the adaptive control of the amount of light entering.

[0020] Preferably, the process of foreign body detection and processing includes:

[0021] The image area from (0, H / 2) to (W, H) is selected as the region of interest, and the pixels outside the region are directly used as the background; in the RGB color space of the region of interest, the foreign matter and the background are modeled with a full covariance mixed Gaussian model with K Gaussian components, and the energy function is established:

[0022] E(β,i,δ,z)=Q(β,i,δ,z)+R(β,z) (1)

[0023] Where β is the pixel opacity, β is 0, which means the current pixel is the background, and β is 1, which means the current pixel is a foreign object; δ represents the grayscale histogram of foreign objects and background, z is the image grayscale value array, z=(z1,…,z n ,…,z N );

[0024] Q(β,i,p,z) is a regional term, which represents the penalty for a pixel being classified as a foreign object or background, and is defined as:

[0025]

[0026] In the formula, D(β n ,i n ,p,z n ) is a mixed Gaussian density model, which is in the form of:

[0027] D(β n ,i n ,δ,z n )=-logπ(β n ,in )-logp(z n |β n ) (3)

[0028] In the formula, p(z n |β n ) is the Gaussian probability distribution π(β n ,i n ) is the mixing weight coefficient, satisfying:

[0029] And 0≤π i ≤1 (4)

[0030] The expression of the mixed Gaussian density model is derived as follows:

[0031]

[0032] Among them, the parameters of the mixed Gaussian model are: the weight π of each Gaussian component, the mean vector μ of each Gaussian component and the covariance matrix; the expression is:

[0033] δ={π(β n ,i n ),μ(β n ,i n ),∑(β n ,i n ),i=1,2,…,K} (6)

[0034] R(β,z) is the boundary term of the input image, representing the penalty for discontinuity of neighborhood pixels, and its expression is:

[0035]

[0036] Where, parameter τ is the scaling factor, which is determined by the contrast of the current image; (m,n)∈C, C is the set of adjacent color pairs, γ is a constant value of 50; if two pixels β in the neighborhood m With β n When the binary norm is small, the probability that the two pixels belong to the same foreign object or the same background is large. If the binary norm of the two pixels is large, there is a small probability that the two pixels belong to the same foreign object or the same background. When the image contrast is low, the binary norm of the two pixels || z m -z n || is small, and the scaling factor τ is used to amplify the two-norm between pixels;

[0037] Each pixel is divided into foreign matter or background, and the RGB of pixel n is substituted into each component of the mixed Gaussian model. Pixel n may belong to the Gaussian component i with the highest probability. n ; It is obtained by the following formula:

[0038] i n =arg min D(β n ,i n ,δ,z n ) (8)

[0039] The parameter mean and covariance of the mixed Gaussian model are estimated by the RGB value, and the Gaussian component weight δ of the pixel is determined by the ratio of the number of pixels belonging to the Gaussian component to the total number of pixels:

[0040] δ=arg min Q(β,i,δ,z) (9)

[0041] Establish a connection graph between each pixel and foreign objects and background, calculate the weights of each edge connecting foreign objects and background, and complete the edge segmentation with the minimum energy:

[0042] min min E(β,i,δ,z) (10)

[0043] Formulas (10)-(12) are iterated repeatedly until the energy function converges, and the iteration is stopped. Finally, the pixels of foreign objects and the pixels belonging to the background are obtained. The pixels belonging to foreign objects are set to zero, and the lane line area of ​​interest after removing foreign objects is obtained.

[0044] Preferably, the lane line extraction process includes:

[0045] The lane line region of interest after removing foreign objects is read in grayscale, and the pixel points {m1,m2,…,m N} Perform binary classification according to grayscale similarity, N = W × H represents the total number of pixels; randomly set the centroids λ0 and λ1 of the two types of pixels, and calculate the binary norm || m between each pixel and the centroid n -λ k ||, where n∈[0,N / 2], k∈{0,1}. Then, select:

[0046]

[0047] As the loss function of pixel classification, the appropriate cluster center is solved through iteration to divide the pixels in the image into two categories; where:

[0048]

[0049] y nk Indicates whether the current pixel belongs to this class. If the Euclidean distance from the centroid of a class is the smallest, the value is 1, otherwise the value is 0, indicating that the pixel does not belong to this class.

[0050] The iteration termination condition is set to make the loss function converge. The iteration is stopped after the termination condition is met. The pixels of lane lines and non-lane lines are distinguished, and the lane lines are adaptively extracted.

[0051] Preferably, the image size acquired by the visual sensor is set to 640*480, and the grayscale range is [0,7].

[0052] The present invention also relates to a lane line automatic extraction system based on a region of interest, comprising:

[0053] Light input control module: used to collect images and adjust the adjustment parameters according to the image's lighting scene and the mapping relationship obtained by pre-training on lighting suitability;

[0054] Foreign body detection and processing module: used to select the region of interest from the image, establish an energy function and perform loop iteration to solve the pixel combination of the foreign body, set the pixels belonging to the foreign body to zero, and obtain the region of interest after removing the foreign body;

[0055] Lane line extraction module: It is used to read the grayscale of the area of ​​interest after removing foreign objects, perform binary classification according to grayscale similarity, set the loss function convergence and termination conditions, stop iteration after the termination conditions are met, complete the pixel distinction between lane lines and non-lane lines, and finally extract the lane lines.

[0056] With this system, autonomous vehicles can perceive the light intensity in the environment in real time and adjust the amount of light entering, avoiding manual modification of the threshold. At the same time, foreign objects within the lane line range are detected and processed to avoid classifying the pixel set of foreign objects as lane lines. The processed pixels are classified and the lane lines are extracted at the pixel level. This solves the shortcomings of traditional methods that are greatly affected by light intensity when extracting lane lines, and deep learning methods that require a lot of prior information when extracting lane lines.

[0057] The substantial effects of the present invention include:

[0058] By performing nonlinear transformation on the detected image and establishing a mapping between different gray levels, the effect of adjusting the pupil diameter of the animal and thus controlling the amount of light entering can be achieved.

[0059] The nonlinear transformation parameters of autonomous vehicles in different lighting scenarios are trained to enable them to achieve adaptive adjustment under various lighting conditions without the need to manually set the lane segmentation threshold.

[0060] Foreign objects in the lane line area are detected and the corresponding pixels are cleared to avoid the negative impact of foreign objects on lane line extraction during autonomous driving, thereby ensuring the accuracy and safety of the extracted lane lines.

[0061] The pixels in the region of interest of the image are classified to complete the classification of lane line and non-lane line pixels, so as to adaptively extract the pixels corresponding to the lane line. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 is a flow chart of an embodiment of the present invention;

[0063] Figure 2 4 is a flow chart of foreign body detection and processing according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution will be clearly and completely described below in combination with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0065] It should be understood that in various embodiments of the present invention, the size of the sequence number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0066] It should be understood that in the present invention, "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0067] The technical solution of the present invention is described in detail with specific embodiments below. The embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0068] Example:

[0069] A method and system for automatically extracting lane lines based on an area of ​​interest, the system includes an incoming light control module, a foreign object detection and processing module, and a lane line extraction module. In this embodiment, the Songling SCOUT_MINI mobile robot is used as an autonomous driving chassis equipped with a visual sensor to build an experimental platform. The method includes the following steps: Figure 1 The following steps are shown:

[0070] Light input control: collect images and adjust the adjustment parameters according to the image's lighting scene and the mapping relationship obtained through pre-training on lighting suitability;

[0071] Foreign body detection and processing: select the region of interest from the image, establish an energy function and perform loop iteration to solve the pixel combination of the foreign body, set the pixels belonging to the foreign body to zero, and obtain the region of interest after removing the foreign body;

[0072] Lane line extraction: After removing foreign objects, the region of interest is read in grayscale, and binary classification is performed according to grayscale similarity. The loss function convergence and termination conditions are set. After the termination conditions are met, the iteration is stopped, and the pixel distinction between lane lines and non-lane lines is completed, and finally the lane lines are extracted.

[0073] Specifically include:

[0074] Step 1: Light adaptation training

[0075] Receive the image information obtained by the visual sensor on the experimental platform, adjust the size to 640*480, and the grayscale range is [0,7]. When the experimental platform is driving, the traffic signs in the current environment are extracted through the detection process in the autonomous driving task, and the pixels of each similar traffic sign are selected as the region of interest (ROI) of the image, read in grayscale, and calculate the grayscale distribution of the ROI under various lighting environments:

[0076] p(s k ) = g k (13)

[0077] The RGB image acquired by the visual sensor is transformed nonlinearly to complete the mapping of grayscale to high grayscale or high grayscale to low grayscale, in the form of:

[0078] p′=cp α (14)

[0079] For g under different lighting scenes k , select different c and α to make the lane line fully extracted and establish g k The mapping f(g k )→c,α, completing the illumination adaptation training.

[0080] Step 2: Adjust the amount of light entering

[0081] In the actual autonomous driving scenario, the ROI of the identified traffic signs is also extracted, and the histogram distribution g of the ROI is calculated. k . Using the trained mapping relationship f(g k )→c,α, automatically selects c and α parameters suitable for the current lighting scene, completes adaptive control of the amount of light entering, and can adaptively suppress the influence of lighting.

[0082] Step 3: Foreign matter detection and treatment, such as Figure 2 As shown:

[0083] The image area from (0, 240) to (640, 240) is selected as the ROI, and the pixels outside the area are directly used as the background. In the RGB color space of the ROI, the foreign matter and the background are modeled with a full covariance mixed Gaussian model with 5 Gaussian components. Establish the energy function:

[0084] E(β,i,δ,z)=Q(β,i,δ,z)+R(β,z) (15)

[0085] Where β is the pixel opacity. When β is 0, it means the current pixel is the background. When β is 1, it means the current pixel is a foreign object. δ represents the grayscale histogram of foreign objects and background. z is the grayscale value array of the image. z=(z1,…,z n ,…,z N ).

[0086] Q(β,i,p,z) is a regional term, which represents the penalty for a pixel being classified as a foreign object or background, and is defined as:

[0087]

[0088] In the formula, D(β n ,i n ,p,z n ) is a mixed Gaussian density model, which is in the form of:

[0089] D(β n ,i n ,δ,z n )=-logπ(β n ,i n )-logp(z n |β n ) (17)

[0090] In the formula, p(z n |β n ) is the Gaussian probability distribution π(β n ,i n ) is the mixing weight coefficient, satisfying:

[0091] And 0≤π i ≤1 (18)

[0092] The expression of the mixed Gaussian density model is derived as follows:

[0093]

[0094] The parameters of the mixed Gaussian model are: the weight of each Gaussian component, the mean vector of each Gaussian component, and the covariance matrix. The expression is:

[0095] δ={π(β n ,i n ),μ(β n ,i n ),∑(β n ,i n ),i=1,2,…,K} (20)

[0096] R(β,z) is the boundary term of the input image, representing the penalty for discontinuity of neighborhood pixels, and its expression is:

[0097]

[0098] Where the parameter τ is the scaling factor, which is determined by the contrast of the current image. m With β n If the binary norm is small, the probability that the two pixels belong to the same foreign object or the same background is high. If the binary norm of the two pixels is large, there is a small probability that the two pixels belong to the same foreign object or the same background. m -z n || is smaller, and the scaling factor τ is used to amplify the two-norm between pixels.

[0099] Each pixel is classified as a foreign object or background, and the RGB of pixel n is substituted into each component of the mixed Gaussian model. Pixel n may belong to the Gaussian component with the highest probability.

[0100] i n =arg min D(β n ,i n ,δ,z n ) (twenty two)

[0101] Then the parameter mean and covariance of the mixed Gaussian model are estimated by the RGB value, and the Gaussian component weight of the pixel is determined by the ratio of the number of pixels belonging to the Gaussian component to the total number of pixels:

[0102] δ=arg min Q(β,i,δ,z) (23)

[0103] Establish a connection graph between each pixel and foreign objects and background, calculate the weights of each edge connecting foreign objects and background, and complete the edge segmentation with the minimum energy:

[0104] min min E(β,i,δ,z) (24)

[0105] Formulas (10)-(12) are iterated repeatedly until the energy function converges, and the iteration is stopped. Finally, the pixels of foreign objects and pixels belonging to the background are obtained. The pixel values ​​of foreign objects are subtracted from the original ROI, that is, the pixels belonging to foreign objects are set to zero, and the lane line ROI after removing foreign objects is obtained.

[0106] Step 4: Lane line extraction

[0107] After removing foreign objects, the lane line ROI is read in grayscale, and each pixel point {m1,m2,…,m N}Based on the grayscale similarity, N=640*480 represents the total number of pixels. First, randomly set the centroids of the two types of pixels λ0,λ1, and calculate the binary norm of each pixel and the centroid ||m n -λ k ||, where n∈[0,N / 2], k∈{0,1}. Then, select

[0108] As the loss function of pixel classification, the appropriate cluster center is solved through iteration to divide the pixels in the image into two categories.

[0109]

[0110] y nk Indicates whether the current pixel belongs to this class. If the Euclidean distance from the centroid of a class is the smallest, the value is 1, otherwise the value is 0, indicating that this pixel does not belong to this class. Finally, set the iteration termination condition to make the loss function converge, and stop the iteration after the termination condition is met. Finally, the pixels belonging to the lane line and the non-lane line can be distinguished, and the autonomous extraction of the lane line can be completed.

[0111] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the specific device can be divided into different functional modules to complete all or part of the functions described above.

[0112] In the embodiments provided in the present application, it should be understood that the disclosed structures and methods can be implemented in other ways. For example, the embodiments of the structure described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another structure, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, structures or units, which can be electrical, mechanical or other forms.

[0113] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0114] In addition, each functional unit in the embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0115] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0116] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for automatically extracting lane lines based on a region of interest, characterized in that: The following steps are involved: Light input control: collect images and adjust the adjustment parameters according to the lighting scene of the image and the mapping relationship obtained through pre-training on lighting suitability; Foreign body detection and processing: select the region of interest from the image, establish an energy function and perform loop iteration to solve the pixel combination of the foreign body, set the pixels belonging to the foreign body to zero, and obtain the region of interest after removing the foreign body; Lane line extraction: After removing foreign objects, the region of interest is read in grayscale, and binary classification is performed according to grayscale similarity. The loss function convergence and termination conditions are set. After the termination conditions are met, the iteration is stopped, and the pixel distinction between lane lines and non-lane lines is completed, and finally the lane lines are extracted; The illumination suitability training process includes: setting the image size acquired by the visual sensor to W×H, the grayscale range to [0, L-1], acquiring the traffic sign bounding box under different illumination scenes, and selecting the pixels in the bounding box as the region of interest, selecting adjustment parameters under different illumination scenes to extract lane lines, and thereby establishing a mapping relationship between the adjustment parameters and the illumination scene; the selecting adjustment parameters under different illumination scenes to extract lane lines, and thereby establishing a mapping relationship between the adjustment parameters and the illumination scene, includes: Calculate the grayscale distribution of each similar region of interest under various lighting environments: p(s k )=g k ; Among them, s k Indicates that a pixel belongs to the kth gray level, g k Represents the total number of pixels belonging to gray level k in the image. The acquired RGB image is transformed nonlinearly to complete the mapping from gray level to high gray level or from high gray level to low gray level. Its form is: p'=cp α ; Where p is the input grayscale of the current image, p′ is the output grayscale of the current image, c is the grayscale scaling ratio, and α is the grayscale nonlinear transformation factor. k , select different c and α, extract the complete lane line, and establish g k The mapping f(g k )→c,α, completing the illumination adaptation training.

2. The method for automatically extracting lane lines based on a region of interest according to claim 1, characterized in that: The process of controlling the amount of light entering includes: Extract the region of interest from the image and calculate the histogram distribution g k , using the trained mapping relationship f(g k )→c,α, select the c and α parameters suitable for the current lighting scene to complete the adaptive control of the amount of light entering.

3. The method for automatically extracting lane lines based on a region of interest according to claim 1 or 2, characterized in that: The process of foreign body detection and processing includes: selecting the image area from (0, H / 2) to (W, H) as the region of interest, and directly taking the pixels outside the area as the background; in the RGB color space of the region of interest, respectively modeling the foreign body and the background with a full covariance mixed Gaussian model with K Gaussian components, and establishing an energy function: E(β,i,δ,z)=Q(β,i,δ,z)+R(β,z) (1) Where β is the pixel opacity. When β is 0, it means the current pixel is the background. When β is 1, it means the current pixel is a foreign object. δ represents the grayscale histogram of foreign objects and background. z is the grayscale value array of the image. z=(z1,L,z n ,L,z N ); Q(β,i,p,z) is a regional term, which represents the penalty for a pixel being classified as a foreign object or background, and is defined as: In the formula, D(β n ,i n ,p,z n ) is a mixed Gaussian density model, which is in the form of: D(β n ,and n ,δ,z n )=-logπ(β n ,and n )-logp(from n |β n ) (3) In the formula, p(z n |β n ) is the Gaussian probability distribution π(β n ,i n ) is the mixing weight coefficient, satisfying: The expression of the mixed Gaussian density model is derived as follows: Among them, the parameters of the mixed Gaussian model are: the weight π of each Gaussian component, the mean vector μ of each Gaussian component and the covariance matrix; the expression is: δ={π(β n ,i n ),μ(b n ,i n ),∑(β n ,i n ),i=1,2,L,K} (6) R(β,z) is the boundary term of the input image, representing the penalty for discontinuity of neighborhood pixels, and its expression is: Where, parameter τ is the scaling factor, which is determined by the contrast of the current image; (m,n)∈C, C is the set of adjacent color pairs, γ is a constant value of 50; if two pixels β in the neighborhood m With β n When the binary norm is small, the probability that the two pixels belong to the same foreign object or the same background is large. If the binary norm of the two pixels is large, there is a small probability that the two pixels belong to the same foreign object or the same background. When the image contrast is low, the binary norm of the two pixels || z m -z n || is small, and the scaling factor τ is used to amplify the two-norm between pixels; Each pixel is divided into foreign matter or background, and the RGB of pixel n is substituted into each component of the mixed Gaussian model. Pixel n may belong to the Gaussian component i with the highest probability. n ; It is obtained by the following formula: and n =arg min D(β n ,and n ,δ,z n ) (8) The parameter mean and covariance of the mixed Gaussian model are estimated by the RGB value, and the Gaussian component weight δ of the pixel is determined by the ratio of the number of pixels belonging to the Gaussian component to the total number of pixels: δ=arg min Q(β,i,δ,z) (9) Establish a connection graph between each pixel and foreign objects and background, calculate the weights of each edge connecting foreign objects and background, and complete the edge segmentation with the minimum energy: min min E(β,i,δ,z) (10) Formulas (10)-(12) are iterated repeatedly until the energy function converges, and the iteration is stopped. Finally, the pixels of foreign objects and the pixels belonging to the background are obtained. The pixels belonging to foreign objects are set to zero, and the lane line area of ​​interest after removing foreign objects is obtained.

4. The method for automatically extracting lane lines based on a region of interest according to claim 3, characterized in that: The process of lane line extraction includes: The lane line area of ​​interest after removing foreign objects is read in grayscale, and the pixel points {m1,m2,L,m N } Perform binary classification according to grayscale similarity, N = W × H represents the total number of pixels; randomly set the centroids λ0 and λ1 of the two types of pixels, and calculate the binary norm || m between each pixel and the centroid n -λ k ||, where n∈[0,N / 2], k∈{0,1}. Then, select: As the loss function of pixel classification, the appropriate cluster center is solved through iteration to divide the pixels in the image into two categories; where: y nk Indicates whether the current pixel belongs to this class. If the Euclidean distance from the centroid of a class is the smallest, the value is 1, otherwise the value is 0, indicating that the pixel does not belong to this class. The iteration termination condition is set to make the loss function converge. The iteration is stopped after the termination condition is met. The pixels of lane lines and non-lane lines are distinguished, and the lane lines are adaptively extracted.

5. The method for automatically extracting lane lines based on a region of interest according to claim 1, characterized in that: Set the image size acquired by the visual sensor to 640*480 and the grayscale range to [0,7].

6. A lane line automatic extraction system based on a region of interest, characterized in that: include: Light input control module: used to collect images and adjust the adjustment parameters according to the image's lighting scene and the mapping relationship obtained by pre-training on lighting suitability; Foreign body detection and processing module: used to select the region of interest from the image, establish an energy function and perform loop iteration to solve the pixel combination of the foreign body, set the pixels belonging to the foreign body to zero, and obtain the region of interest after removing the foreign body; Lane line extraction module: used to read the grayscale of the region of interest after removing foreign objects, perform binary classification according to grayscale similarity, set the loss function convergence and termination conditions, stop iteration after the termination conditions are met, complete the pixel distinction between lane lines and non-lane lines, and finally extract the lane lines; The process of illumination suitability training performed by the light input control module includes: setting the image size acquired by the visual sensor to W×H, the grayscale range to [0, L-1], acquiring the traffic sign bounding box under different illumination scenes, and selecting the pixels in the bounding box as the region of interest, selecting adjustment parameters under different illumination scenes to extract lane lines, and establishing a mapping relationship between the adjustment parameters and the illumination scenes accordingly; The step of selecting adjustment parameters to extract lane lines under different lighting scenes and establishing a mapping relationship between the adjustment parameters and the lighting scenes accordingly includes: Calculate the grayscale distribution of each similar region of interest under various lighting environments: p(s k )=g k ; Among them, s k Indicates that a pixel belongs to the kth gray level, g k Represents the total number of pixels belonging to gray level k in the image. The acquired RGB image is transformed nonlinearly to complete the mapping from gray level to high gray level or from high gray level to low gray level. Its form is: p'=cp α ; Where p is the input grayscale of the current image, p′ is the output grayscale of the current image, c is the grayscale scaling ratio, and α is the grayscale nonlinear transformation factor. k , select different c and α, extract the complete lane line, and establish g k The mapping f(g k )→c,α, completing the illumination adaptation training.

Images