A lane-line-based intelligent driving method and related equipment

Abstract

The present invention discloses an intelligent driving method based on lane lines and related equipment. The method includes: obtaining target road image information, wherein the target image information is road image information of the vehicle's driving direction; obtaining target lane line pixel information based on the target road image information; generating a steering control instruction through the lane line pixel information; and controlling the target vehicle to drive based on the steering control instruction. The method provided in the embodiment of the present application only needs to obtain the target road image information in front of the vehicle, identify the lane line pixel information in the target road image information, and generate a steering control instruction based on the lane line pixel information, thereby controlling the vehicle to drive based on the change of the lane line. The method does not require a large number of sensors to obtain information, and only requires one camera to complete it. The method is simple and effective, with high accuracy, simple hardware conditions, and easy to implement.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving, and more specifically, to a lane-line-based intelligent driving method and related equipment. Background Art

[0002] Lane tracking is a technology in the field of intelligent driving. This technology uses sensors on the vehicle to detect the vehicle status and road information, and then calculates the control signal to the vehicle steering system to enable the vehicle to automatically drive along the lane without human control. As a basic technology in autonomous driving technology, lane tracking can free the driver's hands during driving, improving driving convenience.

[0003] A common method for intelligent driving control based on lane lines is to use an on-board camera to detect the lane lines on both sides of the road, and calculate the driving path that the vehicle should follow based on the distance between the two lane lines and the curvature of the lane lines. Combined with the vehicle status data obtained from the in-vehicle sensors (such as IMU, GPS, steering wheel angle sensor), including longitudinal speed, lateral speed, heading angle, wheel angle, vehicle position, etc., the translation and / or angle difference between the actual path of the vehicle and the ideal driving path is calculated, and then a linear or nonlinear controller is used to drive the vehicle to approach the ideal driving path. This method can often achieve good control effects on vehicles equipped with ADAS systems (Advanced Driving Assistance System) above the L2 level. However, in scenarios such as unmanned warehouse trucks and unmanned shuttles in parks, the vehicle may not be equipped with a full set of sensor equipment due to restrictions such as cost and measurement conditions. The above method is too costly and has poor economic efficiency. Summary of the invention

[0004] A series of simplified concepts are introduced in the Summary of the Invention, which will be further described in detail in the Detailed Description of the Invention. The Summary of the Invention does not mean to attempt to define the key features and essential technical features of the claimed technical solution, nor does it mean to attempt to determine the scope of protection of the claimed technical solution.

[0005] In order to provide an intelligent driving method with strong economic efficiency, in a first aspect, the present invention proposes an intelligent driving method based on lane lines, the method comprising:

[0006] Acquire target road image information, wherein the target image information is road image information in the direction of vehicle travel;

[0007] Acquire target lane line pixel information based on the target road image information;

[0008] Generate a steering control instruction based on the lane line pixel information;

[0009] The target vehicle is controlled to travel based on the steering control instruction.

[0010] Optionally, the target road image includes a target road left image and a target road right image, and the target lane line pixel information includes current left lane line pixel information and current right lane line pixel information;

[0011] The above-mentioned obtaining lane line pixel information based on the above-mentioned target road image information includes:

[0012] Obtaining the left lane line pixel information in the left side image of the target road;

[0013] Obtaining the right lane line pixel information in the right side image of the target road;

[0014] The above-mentioned generating a steering control instruction through the lane line pixel information includes:

[0015] Based on the above current left lane line pixel information, the current left lane cumulative pixel value is obtained;

[0016] Obtain the right turn angle according to the above current left lane cumulative pixel value;

[0017] Based on the above current right lane line pixel information, the current right lane cumulative pixel value is obtained;

[0018] Obtain the left turn angle according to the above current right lane cumulative pixel value;

[0019] The right turn angle and the left turn angle are summed to obtain a first turning instruction;

[0020] The first steering instruction is determined as the steering control instruction.

[0021] Optionally, the above method further includes:

[0022] The target road image is divided according to a preset ratio to obtain the target road left image and the target road right image.

[0023] Optionally, the above method further includes:

[0024] In the case where there are a plurality of left lane lines in the left side image of the target road, obtaining pixel information of the current left lane line corresponding to the left lane line close to the left side;

[0025] or

[0026] In the case where there are a plurality of right lane lines in the right side image of the target road, pixel information of the current right lane line corresponding to the right lane line close to the right side is obtained.

[0027] Optionally, the above-mentioned target lane line pixel information also includes image speed information of the straight left lane line;

[0028] The above-mentioned generating a steering control instruction through the lane line pixel information includes:

[0029] When the definition of the right side image of the target road is less than the preset definition, the pixel value of the left straight lane line is obtained according to the pixel information of the left straight lane line;

[0030] Obtaining a first turning instruction according to the difference between the current left lane accumulated pixel value and the left lane line pixel value;

[0031] The first steering instruction is determined as the steering control instruction.

[0032] Optionally, the above-mentioned target lane line pixel information also includes image speed information of the straight right lane line;

[0033] The above-mentioned generating a steering control instruction through the lane line pixel information includes:

[0034] When the definition of the left side image of the target road is less than the preset definition, the pixel value of the straight right lane line is obtained according to the pixel information of the straight right lane line;

[0035] Obtaining a second turning instruction according to the difference between the above-mentioned current right lane accumulated pixel value and the above-mentioned straight right lane line pixel value;

[0036] The second steering instruction is determined as the steering control instruction.

[0037] Optionally, the above method further includes:

[0038] Get original image information;

[0039] Extracting road image information based on the original image information;

[0040] Performing a convolution operation on the road image information to obtain denoised image information;

[0041] Performing edge extraction operation on the above-mentioned denoised image information to obtain characteristic edge image information;

[0042] A color filtering operation is performed on the characteristic edge image information to obtain the target road image information.

[0043] In a second aspect, the present invention further proposes an intelligent driving control device based on lane lines, comprising:

[0044] A first acquisition unit is used to acquire target road image information, wherein the target image information is road image information in the direction of vehicle travel;

[0045] A second acquisition unit, used to acquire target lane line pixel information based on the target road image information;

[0046] A generating unit, used to generate a steering control instruction according to the lane line pixel information;

[0047] A control unit is used to control the driving of the target vehicle based on the above steering control instructions.

[0048] In a third aspect, an electronic device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is used to implement the steps of the lane line-based intelligent driving method of any one of the first aspects described above when executing the computer program stored in the memory.

[0049] In a fourth aspect, the present invention further proposes a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the lane-line-based intelligent driving method of any one of the above items in the first aspect is implemented.

[0050] In summary, the lane line-based intelligent driving method proposed in the embodiment of the present application includes: obtaining target road image information, wherein the target image information is the road image information of the vehicle's driving direction; obtaining target lane line pixel information based on the target road image information; generating a steering control instruction through the lane line pixel information; and controlling the target vehicle to drive based on the steering control instruction. The method provided in the embodiment of the present application only needs to obtain the target road image information in front of the vehicle, identify the lane line pixel information in the target road image information, and generate a steering control instruction based on the lane line pixel information, thereby controlling the vehicle to drive based on the change of the lane line. The method does not require a large number of sensors to obtain information, and only requires one camera to complete. The method is simple and effective, with high accuracy, simple hardware conditions, and easy to implement.

[0051] The lane-line-based intelligent driving method of the present invention, and other advantages, objectives, and features of the present invention will be partially reflected through the following description, and will also be partially understood by those skilled in the art through research and practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present specification. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0053] Figure 1A schematic diagram of a process flow of an intelligent driving method based on lane lines provided in an embodiment of the present application;

[0054] Figure 2 A target road schematic diagram provided in an embodiment of the present application;

[0055] Figure 3 A schematic diagram of a target straight road provided in an embodiment of the present application;

[0056] Figure 4 A schematic diagram of a target left-turn road provided in an embodiment of the present application;

[0057] Figure 5 A schematic diagram of a target right-turn road provided in an embodiment of the present application;

[0058] Figure 6 A schematic diagram of a simulation test result provided in an embodiment of the present application;

[0059] Figure 7 A schematic diagram of simulation test parameters provided in an embodiment of the present application;

[0060] Figure 8 A parameter description schematic diagram provided for an embodiment of the present application;

[0061] Fig. 9 An intelligent driving control device based on lane lines provided in an embodiment of the present application;

[0062] Fig.10 A schematic diagram of the structure of a lane-line-based intelligent driving electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] The method provided in the embodiment of the present application only needs to obtain the target road image information in front of the vehicle, identify the lane line pixel information in the target road image information, and generate a steering control instruction based on the lane line pixel information, so as to control the vehicle to travel based on the change of the lane line. This method does not require a large number of sensors to obtain information, and only requires one camera to complete it. In addition, the method is simple and effective, with high accuracy, simple hardware conditions, and easy to implement.

[0064] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0065] See also Figure 1 , is a schematic flow chart of a lane-based intelligent driving method provided in an embodiment of the present application, which may specifically include:

[0066] S110, acquiring target image information, wherein the target image information is road image information in the direction of vehicle travel;

[0067] For example, a common method for intelligent driving control based on lane lines is to use a vehicle-mounted camera to detect the lane lines on both sides of the road, and calculate the driving path that the vehicle should follow based on the distance between the two lane lines and the curvature of the lane lines. Combined with the vehicle status data obtained from the in-vehicle sensors (such as IMU, GPS, steering wheel angle sensor), including longitudinal speed, lateral speed, heading angle, wheel angle, vehicle position, etc., the translation and / or angle difference between the actual path of the vehicle and the theoretical driving path is calculated, and then a linear or nonlinear controller is used to drive the vehicle to approach the theoretical driving path. This method can often achieve good control effects on vehicles equipped with L2 or higher level ADAS systems. However, in scenarios such as unmanned trucks in warehouses and unmanned shuttles in parks, the vehicle may not be equipped with a full set of sensing equipment due to restrictions such as cost and measurement conditions. The above method is too costly and has poor economic efficiency.

[0068] This application only needs to obtain the road image information of the vehicle's driving direction, and after corresponding post-image processing, the vehicle's steering information can be obtained. First, the image information of the vehicle's driving direction, that is, the target image information, is obtained through a camera installed at the front end of the vehicle. It is understandable that the camera can be installed in the vehicle's grille, the front windshield, or other unobstructed locations at the front of the vehicle. The images obtained by the camera may include images other than the road, such as distant views such as the sky and mountains, and only the road image information can be obtained through preprocessing.

[0069] S120, obtaining target lane line pixel information through the above target image information;

[0070] Exemplarily, lane line information is extracted from the target image information. The method for extracting the lane line can be based on the contour shape or based on the image color difference. It can be understood that the color of the lane line and the color of the road will form a color difference. The target image can be pre-processed accordingly, and the lane line pixel information can be extracted by using a suitable threshold.

[0071] S130, generating a steering control instruction according to the lane line pixel information;

[0072] For example, when the road is straight, the lane line pixels appearing in the camera should remain unchanged, but when turning, the lane line image captured by the camera will bend to the left or right, such as Figure 2 As shown, this is the lane line image obtained when the vehicle encounters a left turn intersection. The lane line pixel information will change, increase or decrease, and the lane line pixel information change will have a certain relationship with the turning radius. The lane line pixel information is used to generate a steering control instruction.

[0073] S140: Control the target vehicle to travel based on the steering control instruction.

[0074] Exemplarily, according to the steering control instruction generated by the above steps, the steering angle of the vehicle is obtained, and the vehicle driving is controlled according to the steering angle, thereby ensuring that the vehicle drives along the middle of the two lanes, realizing intelligent driving.

[0075] In summary, the method provided in the embodiment of the present application only needs to obtain the target road image information in front of the vehicle, identify the lane line pixel information in the target road image information, and generate a steering control instruction based on the lane line pixel information, so as to control the vehicle to travel based on the changes in the lane line. This method does not require a large number of sensors to obtain information, and only requires one camera to complete it. In addition, the method is simple and effective, with high accuracy, simple hardware conditions, and easy implementation.

[0076] Optionally, the target road image includes a target road left image and a target road right image, and the target lane line pixel information includes current left lane line pixel information and current right lane line pixel information;

[0077] The above-mentioned obtaining lane line pixel information based on the above-mentioned target road image information includes:

[0078] Obtaining the left lane line pixel information in the left side image of the target road;

[0079] Obtaining the right lane line pixel information in the right side image of the target road;

[0080] The above-mentioned generating a steering control instruction through the lane line pixel information includes:

[0081] Based on the above current left lane line pixel information, obtain the current left lane cumulative pixel value;

[0082] Obtain a right turn signal according to the above current left lane accumulated pixel value;

[0083] Based on the above current right lane line pixel information, obtain the current right lane cumulative pixel value;

[0084] Obtain a left turn signal according to the above current right lane accumulated pixel value;

[0085] Summing the right turn signal and the left turn signal to obtain a first turning instruction;

[0086] The first steering instruction is determined as the steering control instruction.

[0087] For example, the target road image includes a target road left image and a target road right image. The target road left image and the target road right image can be obtained by dividing the image acquired by a camera according to a certain ratio, or can be acquired by two cameras distributed on both sides of the front of the vehicle. The target lane line pixel information includes the current left lane line pixel information and the current right lane line pixel information. Figure 2The original image shown is a schematic diagram of an international two-way lane. The dotted line in the middle is the left lane line for driving forward, and the solid line on the right is the right lane line for current driving. The left lane line and the right lane line may exist simultaneously in the left image or the right image. However, in the process of obtaining lane line information, only the current left lane information is extracted in the left image of the target road and the right lane is ignored. In the right image of the target road, only the current right lane information is extracted and the left lane is ignored. The current left lane cumulative pixel value and the current right lane pixel value are obtained based on the current left lane pixel information and the current right lane pixel information, respectively. The right turn angle signal is obtained based on the current left lane pixel value, and the left turn angle signal is obtained based on the current right lane pixel value. The left turn angle and the right turn angle are summed to obtain the first steering instruction, and the first steering instruction is determined as the above-mentioned steering control instruction. For example: when the road ahead is straight, such as Figure 3 As shown in the figure, the current left lane cumulative pixel value (dashed line) is 50, and the current right lane (solid line) cumulative pixel value is 50. It can be stipulated in advance that the left is positive and the right is negative. The left pixel value generates the right turn angle, and the right pixel value generates the left turn angle, that is, the right turn angle is -50, the left turn angle is +50, and the sum of the right turn angle and the left turn angle is 0. At this time, there is no need to change the driving direction of the vehicle. When the road ahead of the vehicle is turning left, such as Figure 4 As shown in the figure, the current left lane cumulative pixel value (dashed line) is -10, the current right lane (solid line) pixel value is 80, the sum of the right turn angle and the left turn angle is +70, and the left is positive as stipulated above, so the comprehensive turn signal is 70. When the road ahead of the vehicle is turning, such as Figure 4 As shown, the current right lane cumulative pixel value (dashed line) is -90, the current right lane (solid line) pixel value is 10, the sum of the right turn angle and the left turn angle is -80, the above-mentioned right is negative, so the comprehensive turn signal is 80. It should be noted that the above specific values ​​are only for illustration, and the specific steering angle needs to be calibrated according to the installation position of the camera and the total pixel amount of the image. In addition, since the left lane line is a dashed line and the right lane line is a solid line, in the case of straight driving, there is still a phenomenon that the current left lane cumulative pixel value is different from the current right lane pixel value. Through calibration, a suitable adjustment ratio can be set to meet the control requirements.

[0088] In summary, the method provided in this embodiment obtains the current left lane cumulative pixel value in the left image of the target road, obtains the current right lane cumulative pixel value in the right image of the target road, and forms right / left turn angle signals respectively through the current left / right lane cumulative pixel values, and obtains the first steering instruction through the sum of the two angle signals. This method obtains the steering instruction of the vehicle based on the accumulation of the left and right lane lines. The method is simple, effective, and easy to implement in engineering.

[0089] Optionally, the above method further includes:

[0090] The target road image is divided according to a preset ratio to obtain the target road left image and the target road right image.

[0091] For example, Figure 2 The original image shown is a schematic diagram of an international two-way lane. The dotted line in the middle is the left lane line for driving forward, and the solid line on the right is the right lane line for current driving. After preprocessing the original image, the target road image can be obtained. The target road image is divided into the left image of the target road and the right image of the target road according to a preset ratio (1:1 can be selected), and the current left lane line information and the current right lane line information are identified based on the above method, and control instructions are obtained to control the intelligent driving of the vehicle. According to this method, only one camera needs to be installed in front of the vehicle. By dividing the acquired picture into the left image and the right image according to a preset ratio, and based on the above lane line recognition method, the intelligent driving of the vehicle is controlled, which reduces the hardware cost to a certain extent.

[0092] In summary, the method provided in the embodiment of the present application can obtain an original image through a monocular camera, and pre-process and divide the original image at a preset ratio to obtain a left side image of the target road and a right side image of the target road. Based on the left side image of the target road and the right side image of the target road, the left / right lane line information is obtained, and the left / right turning angle signal is calculated according to the left / right lane line information, so as to generate a control instruction to control the vehicle driving. The method only requires a monocular camera as hardware, which can effectively achieve the purpose of reducing costs.

[0093] Optionally, the above method further includes:

[0094] In the case where there are a plurality of left lane lines in the left side image of the target road, obtaining pixel information of the current left lane line corresponding to the left lane line close to the left side;

[0095] or

[0096] In the case where there are a plurality of right lane lines in the right side image of the target road, pixel information of the current right lane line corresponding to the right lane line close to the right side is obtained.

[0097] For example, Figure 5As shown in the figure, two solid lines (i.e., right lane lines) appear in the right image of the target. If the sum of the pixel information of the two right lane lines is used as the current right lane line pixel information, it will cause a phenomenon that does not conform to the actual situation, because the longer lane line above is the right lane line of the oncoming vehicle. At this time, the short solid line near the lower right should be selected as the current right lane line, and its pixel information should be obtained for vehicle intelligent driving. Correspondingly, when there are multiple left lane lines in the left image of the target, the left lane line near the left side is obtained as the current left lane line. It can be understood that when there are multiple lane lines, a reasonable distance threshold can be set for screening to prevent the oncoming lane line from affecting it.

[0098] In summary, the method provided in this embodiment can effectively screen the opposite lane line when multiple lane lines appear in an image on one side, thereby avoiding logical confusion caused by control behavior.

[0099] Optionally, the above-mentioned target lane line pixel information also includes image speed information of the straight left lane line;

[0100] The above-mentioned generating a steering control instruction through the lane line pixel information includes:

[0101] When the definition of the right side image of the target road is less than the preset definition, the pixel value of the left straight lane line is obtained according to the pixel information of the left straight lane line;

[0102] Obtaining a first turning instruction according to the difference between the current left lane accumulated pixel value and the left lane line pixel value;

[0103] The first steering instruction is determined as the steering control instruction.

[0104] Exemplarily, when the clarity of the right image of the target road is less than the preset clarity, it is impossible to use the right image and the left image to simultaneously generate the steering angle to obtain the steering instruction. At this time, the current pixel value of the left lane line is obtained, and the first steering instruction is obtained by the difference between the current pixel of the left lane line and the pixel value of the straight left lane line, and the first steering instruction is used as the steering control instruction to control the vehicle. It can be understood that when the lane line ahead is a left turn, the current pixel value of the left lane line will be reduced compared to the cumulative pixel value of the straight left lane, and the left turn control is performed according to the proportional value. Correspondingly, if the lane line ahead is a right turn, the current pixel value of the left lane line will be increased compared to the cumulative pixel value of the straight left lane, and the right turn control is performed according to the difference. It should be noted that the pixel value of the straight left lane line can be set to a fixed value in advance, or it can be based on the value obtained when driving straight.

[0105] In summary, the method provided in this embodiment avoids safety hazards caused by jointly controlling the vehicle driving based on the lane information of the left and right images when the clarity of the right image is insufficient. At this time, the first steering instruction is generated only based on the difference between the accumulated pixel value of the left lane and the pixel value of the executing left lane line, and the first steering instruction is used as the steering control instruction to control the vehicle to turn, thereby improving the safety performance of vehicle control and avoiding accidents.

[0106] Optionally, the above-mentioned target lane line pixel information also includes image speed information of the straight right lane line;

[0107] The above-mentioned generating a steering control instruction through the lane line pixel information includes:

[0108] When the definition of the left side image of the target road is less than the preset definition, the pixel value of the straight right lane line is obtained according to the pixel information of the straight right lane line;

[0109] Obtaining a second turning instruction according to the difference between the above-mentioned current right lane accumulated pixel value and the above-mentioned straight right lane line pixel value;

[0110] The second steering instruction is determined as the steering control instruction.

[0111] For example, when the clarity of the left image of the target road is less than the preset clarity, it is impossible to use the left image and the right image to simultaneously generate the steering angle to obtain the steering instruction. At this time, the current pixel value of the right lane line is obtained, and the difference between the current pixel value of the right lane line and the pixel value of the straight right lane line is used to obtain the second steering instruction, and the second steering instruction is used as the steering control instruction to control the vehicle. It can be understood that when the lane line ahead is a right turn, the current pixel value of the right lane line will be reduced compared to the cumulative pixel value of the straight right lane, and the right turn control is performed according to the difference. Correspondingly, if the lane line ahead is a left turn, the current pixel value of the right lane line will be increased compared to the cumulative pixel value of the straight right lane, and the left turn control is performed according to the proportional value. It should be noted that the pixel value of the straight right lane line can be set to a fixed value in advance, or it can be based on the value obtained when driving straight.

[0112] In summary, the method provided in this embodiment avoids safety hazards caused by jointly controlling the vehicle driving based on the lane information of the right and left images when the clarity of the left image is insufficient. At this time, the second steering instruction is generated only based on the difference between the accumulated pixel value of the right lane and the pixel value of the executing right lane line, and the second steering instruction is used as the steering control instruction to control the vehicle to turn, thereby improving the safety performance of vehicle control and avoiding accidents.

[0113] In some examples, the method further includes:

[0114] Get original image information;

[0115] Extracting road image information based on the original image information;

[0116] Performing a convolution operation on the road image information to obtain denoised image information;

[0117] Performing edge extraction operation on the above-mentioned denoised image information to obtain characteristic edge image information;

[0118] A color filtering operation is performed on the characteristic edge image information to obtain the target road image information.

[0119] Exemplarily, the image captured directly by the camera is the original image information. Since the original image information may contain noise or other useless information, in order to reduce the impact of these useless information on subsequent lane line recognition and other operations, the original image needs to be preprocessed. The preprocessing can mainly include extracting road image information from the original image information, performing convolution operations on the road information to obtain denoised image information, performing edge extraction operations on the denoised image information to obtain feature edge image information, performing color filtering operations on the feature edge image information to obtain target image information, and using the target image information to perform lane line recognition and control command determination and other tasks.

[0120] In some examples, preprocessing the raw image may include:

[0121] A: Filter out the images above the horizon:

[0122] The intelligent driving method based on lane lines only needs to pay attention to the road conditions. Therefore, in order to reduce the amount of calculation of the processor and graphics card, the part of the picture above the horizon is first removed. In order to achieve this effect, the position of the horizon in the picture must be determined first. The position of the horizon can be dynamically determined according to the pitch angle of the vehicle and the installation position of the camera, or an appropriate fixed value can be selected, that is, the lane line within the position range close to the front of the vehicle can be selected. In this embodiment, the 3 meters in front of the vehicle can be selected as the target range, and the coordinate points (3,0,1) (3,-5,1) of the two horizons are selected in the normalized coordinate system of the three-dimensional space to obtain the horizon equation of the three-dimensional space. By calibrating the camera parameters, the normalized matrix H of the camera can be obtained. H can be used to convert the horizon of the three-dimensional space into the horizon of the two-dimensional image space. The content above this line is invalid information, which can reduce the amount of calculation.

[0123] B: Improve signal-to-noise ratio:

[0124] The images generated by the camera inevitably contain noise, which is generally randomly distributed. Based on this property of noise, combined with the lighting conditions of the road, the resolution of the camera and other conditions, this paper proposes to use a normal distribution convolution kernel with a mean of 0 and a mean square error of 3 to convolve the original image. The processed image is slightly blurred, but most of the background noise is offset.

[0125] C: Edge Extraction:

[0126] Lane lines are line patterns, and sober filtering can be used to extract horizontal and vertical edges in the image. The principle of the sober filter is to extract edge elements through the brightness change gradient of pixel points. Reflective points, stains, and small obstacles on the road surface may also be retained. In order to filter out these contents that are not related to lane lines, it is necessary to calculate the root mean square of the brightness change intensity of the image in the horizontal and vertical directions, and only retain the image content whose root mean square is greater than a preset threshold (10 in this embodiment).

[0127] D: Image split left and right:

[0128] In order to make the monocular camera "simulate" the effect of the binocular camera, the image needs to be divided into two parts. The left image is used to detect the left road line, and the right image is used to detect the right lane line.

[0129] E: Color Filter:

[0130] In order to calculate the actuator excitation intensity for left and right turns respectively, it is also necessary to extract the left lane line and the right lane line of the road in the left image and the right image respectively. In the driving environment of this embodiment, the vehicle is driving on the right side. According to the general marking rules for two-way lanes in the world, the left lane is a yellow dashed line and the right lane is a white solid line. When the target image is in HSV format, the yellow dashed line and the white solid line can be extracted by adjusting the range of H, S, and V values.

[0131] In summary, the method provided in this embodiment can effectively filter out noise and irrelevant information, and accurately and quickly extract the target road image information.

[0132] In some examples, the vehicle model used in the simulation test using the above method has a size ratio of about 20:1 to that of a general passenger car. After calibration and tuning, the vehicle model can achieve a relatively ideal control effect in the simulator and the real environment. Figure 6 and Figure 7 As shown in , the vehicle can smoothly follow the lane line. The vehicle's angular offset and lateral offset are extracted from the vehicle's trajectory, lateral offset, heading angle, and lane line tangent direction angle. It should be noted that Figure 8As shown in , the angle offset is the angle θ between the vehicle's driving direction and the lane line direction, and the lateral offset is the vertical distance d between the vehicle's center position and the left lane line. Figure 7 From the data in , we can see that the average angle deviation of the vehicle's driving path is 3.2 degrees, and the average lateral deviation is 2.7 cm, both of which are within a reasonable range. The above lane-line-based intelligent driving method has high accuracy and can be applied to engineering practice.

[0133] See also Fig. 9 , an embodiment of the lane line-based intelligent driving control device in the embodiment of the present application may include:

[0134] A first acquisition unit 21 is used to acquire target road image information, wherein the target image information is road image information in the direction of vehicle travel;

[0135] A second acquisition unit 22, used to acquire target lane line pixel information based on the target road image information;

[0136] A generating unit 23, configured to generate a steering control instruction according to the lane line pixel information;

[0137] The control unit 24 is used to control the target vehicle to travel based on the above steering control instruction.

[0138] like Fig.10 As shown, an embodiment of the present application also provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 320 and executable on the processor. When the processor 320 executes the computer program 311, the steps of any of the above-mentioned methods of lane-based intelligent driving are implemented.

[0139] Since the electronic device introduced in this embodiment is a device used to implement an intelligent driving control device based on lane lines in the embodiment of the present application, based on the method introduced in the embodiment of the present application, technical personnel in this field can understand the specific implementation mode of the electronic device of this embodiment and its various variations. Therefore, how the electronic device implements the method in the embodiment of the present application is not introduced in detail here. As long as the equipment used by technical personnel in this field to implement the method in the embodiment of the present application is within the scope of protection of this application.

[0140] In the specific implementation process, when the computer program 311 is executed by the processor, it can achieve Figure 1 Any implementation manner in the corresponding embodiments.

[0141] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0142] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0143] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0144] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0145] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0146] The present application also provides a computer program product, which includes computer software instructions. When the computer software instructions are executed on a processing device, the processing device is caused to execute the following Figure 1 The process of the lane line-based intelligent driving method in the corresponding embodiment.

[0147] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)), etc.

[0148] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0149] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0150] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0151] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0152] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0153] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An intelligent driving method based on lane lines, characterized in that: include: Acquire target road image information, wherein the target road image information is road image information in the vehicle's driving direction, and the target road image information includes a left side image of the target road and a right side image of the target road; Acquire target lane line pixel information based on the target road image information, wherein the target lane line pixel information includes current left lane line pixel information and current right lane line pixel information; Generate a steering control instruction based on the lane line pixel information; Controlling the target vehicle to travel based on the steering control instruction; The acquiring target lane line pixel information based on the target road image information includes: acquiring the current left lane line pixel information in the left image of the target road; acquiring the current right lane line pixel information in the right image of the target road; The method of generating a steering control instruction through the lane line pixel information includes: obtaining a current left lane cumulative pixel value based on the current left lane line pixel information; obtaining a right turn angle signal amplitude based on the current left lane cumulative pixel value; obtaining a current right lane cumulative pixel value based on the current right lane line pixel information; obtaining a left turn angle signal amplitude based on the current right lane cumulative pixel value; summing the right turn angle signal amplitude and the left turn angle signal amplitude to obtain a first steering instruction; and determining the first steering instruction as the steering control instruction.

2. The method according to claim 1, characterized in that Also includes: The target road image is divided according to a preset ratio to obtain the target road left image and the target road right image.

3. The method according to claim 1, characterized in that Also includes: In the case where there are multiple left lane lines in the left side image of the target road, obtaining pixel information of the current left lane line corresponding to the left lane line close to the left side; or In the case where there are multiple right lane lines in the right side image of the target road, the current right lane line pixel information corresponding to the right lane line close to the right side is obtained.

4. The method according to claim 1, characterized in that The target lane line pixel information also includes straight left lane line pixel information; The generating a steering control instruction according to the lane line pixel information includes: When the definition of the right side image of the target road is less than a preset definition, obtaining the accumulated pixel value of the straight left lane line according to the pixel information of the straight left lane line; Acquire a first turning instruction according to the difference between the current left lane cumulative pixel value and the left lane line cumulative pixel value; The first steering instruction is determined as the steering control instruction.

5. The method according to claim 1, characterized in that The target lane line pixel information also includes straight right lane line pixel information; The generating a steering control instruction according to the lane line pixel information includes: When the definition of the left side image of the target road is less than a preset definition, acquiring the accumulated pixel value of the straight right lane line according to the pixel information of the straight right lane line; Acquire a second turning instruction according to the difference between the current right lane cumulative pixel value and the straight right lane line cumulative pixel value; The second steering instruction is determined as the steering control instruction.

6. The method according to claim 1, characterized in that Also includes: Get original image information; Extracting road image information based on the original image information; Performing a convolution operation on the road image information to obtain denoised image information; Performing an edge extraction operation on the denoised image information to obtain characteristic edge image information; A color filtering operation is performed on the characteristic edge image information to obtain the target road image information.

7. An intelligent driving control device based on lane lines, characterized in that: include: A first acquisition unit is used to acquire target road image information, wherein the target road image information is road image information in the vehicle's driving direction, and the target road image information includes a left side image of the target road and a right side image of the target road; A second acquisition unit is used to acquire target lane line pixel information based on the target road image information, wherein the target lane line pixel information includes current left lane line pixel information and current right lane line pixel information; A generating unit, configured to generate a steering control instruction according to the lane line pixel information; A control unit, used for controlling the target vehicle to travel based on the steering control instruction; The second acquisition unit is further used to acquire the pixel information of the current left lane line in the left image of the target road; and acquire the pixel information of the current right lane line in the right image of the target road; The generating unit is further configured to obtain a current left lane cumulative pixel value based on the current left lane line pixel information; obtain a right turn angle signal amplitude based on the current left lane cumulative pixel value; obtain a current right lane cumulative pixel value based on the current right lane line pixel information; obtain a left turn angle signal amplitude based on the current right lane cumulative pixel value; obtain a first steering instruction by summing the right turn angle signal amplitude and the left turn angle signal amplitude; and determine the first steering instruction as the steering control instruction.

8. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is used to implement the steps of the lane-line-based intelligent driving method as described in any one of claims 1 to 7 when executing the computer program stored in the memory.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the lane line-based intelligent driving method as described in any one of claims 1 to 7 is implemented.

Images