Lane line track point tracking method, system, equipment and medium

Abstract

The invention provides a lane line track point tracking method, system and device and a medium, and the method comprises the steps: carrying out the lane line detection and coordinate system conversion of a front view image of a vehicle, and obtaining a previous frame lane line point set and a current frame lane line point set under a coordinate system of the vehicle, fitting the current frame lane line point set to obtain a current frame lane line detection equation; estimating the tracking coordinate of each lane line point in the previous frame lane line point set by using the previous frame vehicle motion information according to the previous frame vehicle coordinate system transformation relation to obtain a previous frame lane line track point tracking equation; and fusing the current frame lane line detection equation and the previous frame lane line trajectory point tracking equation to obtain a lane line trajectory point tracking result. According to the method provided by the invention, modeling is carried out based on self-vehicle motion information to realize tracking of lane line trajectory points, and the method is not limited by prior assumptions in an existing lane line tracking method based on image historical frame lane line positions and a lane line parameter filtering mode.

Description

Technical Field

[0001] The present invention belongs to the field of autonomous driving and relates to lane line information acquisition technology, and specifically to a lane line trajectory point tracking method, system, equipment and medium. Background Art

[0002] Lane lines are one of the basic traffic signs and serve as a constraint and guide for car driving. With the development of autonomous driving technology, lane lines are of great significance to auxiliary driving functions such as lane keeping and collision warning, as well as environmental perception to build safe driving areas, making the requirements for lane line tracking accuracy increasingly higher.

[0003] At present, lane line tracking methods include lane line tracking based on lane line position in image history frames and lane line tracking based on lane line parameter filtering. Among them, the lane line tracking method based on lane line position in image history frames relies on the assumption that "there is no large fluctuation in the front and rear frames of the lane line" to track the lane line. However, in complex and changeable road environments, high driving speeds, uncertain steering, or low camera frame rates, the lane line position in the front and rear frame images is likely to fluctuate greatly, and accurate lane line information cannot be obtained by this method. The lane line tracking method based on lane line parameter filtering is greatly affected by the selection of filters. For example, the Kalman filter is more suitable for linear systems and application scenarios where the state probability density conforms to the Gaussian distribution. However, the lane line parameter noise distribution state obtained by general driving scene detection does not always conform to the Gaussian distribution, which makes the filter tracking method inapplicable in some specific scenarios. Others, such as improved Kalman filtering, particle filtering, etc., also have their own limitations, which will affect the accuracy of lane line tracking results.

[0004] In general, the above two lane line tracking methods are difficult to achieve lane line tracking tasks in driving scenarios such as variable vehicle motion and variable road environment. Summary of the invention

[0005] In order to solve the technical problem that the existing lane line tracking method is difficult to achieve the lane line tracking task in driving scenarios such as the ego vehicle motion being changeable and the road environment being changeable, the present invention discloses a lane line trajectory point tracking method based on the ego vehicle motion information. The lane line trajectory point tracking method is to track the lane line trajectory points by modeling the ego vehicle motion information, and is not limited by the prior assumptions in the existing lane line tracking methods based on the lane line position of the image history frame and the lane line parameter filtering method. Specifically, the lane line trajectory point tracking method comprises the following steps:

[0006] S1. Perform lane line detection and coordinate system conversion on the collected front view image of the vehicle to obtain a lane line point set in the vehicle coordinate system, wherein the lane line point set includes a lane line point set of a previous frame and a lane line point set of a current frame, and perform fitting processing on the lane line point set of the current frame to obtain a lane line detection equation of the current frame;

[0007] S2, according to the transformation relationship between the ego-vehicle coordinate systems of the previous and next frames, using the ego-vehicle motion information of the previous frame to estimate the tracking coordinates of each lane line point in the lane line point set of the previous frame, and obtaining the lane line trajectory point tracking equation of the previous frame through the tracking coordinates of each lane line point in the lane line point set of the previous frame;

[0008] S3. The lane line detection equation of the current frame and the lane line trajectory point tracking equation of the previous frame are fused by a given fusion weight to obtain a lane line trajectory point tracking result.

[0009] In an optional embodiment of the above step S1, performing lane line detection and coordinate system conversion on the collected front view image of the vehicle to obtain a lane line point set in the vehicle coordinate system includes:

[0010] S11, using a calibrated front-view camera of the vehicle to collect the front-view image, and using a pre-trained segmentation model or point regression model to perform lane line detection on the front-view image to obtain a 2D image lane line point set;

[0011] S12. According to the pose conversion matrix between the pixel coordinate system and the vehicle coordinate system, the lane line point set of the 2D image is converted into a coordinate system to obtain a lane line point set in the vehicle coordinate system.

[0012] In an optional embodiment of the above step S11, the forward-looking camera is calibrated using a computer vision library or MATLAB camera calibration software to obtain a pose conversion matrix between the pixel coordinate system and the self-vehicle coordinate system, wherein the pose conversion matrix includes a rotation matrix and a translation vector of the forward-looking camera relative to the self-vehicle.

[0013] In an optional embodiment of the above step S2, estimating the tracking coordinates of each lane line point in the lane line point set of the previous frame by using the vehicle motion information of the previous frame according to the transformation relationship between the vehicle coordinate systems of the previous and subsequent frames includes:

[0014] S21, establishing the ego-vehicle coordinate system in the subsequent frame, calculating the new position of a certain point in the ego-vehicle coordinate system in the ego-vehicle coordinate system of the next frame according to the ego-vehicle speed and the rate of change of the heading angle, and obtaining the transformation relationship between the ego-vehicle coordinate systems of the previous and next frames;

[0015] S22. Estimate the tracking coordinates of each lane line point in the previous lane line point set using the transformation relationship through the collected previous frame of ego vehicle motion information, wherein the previous frame of ego vehicle motion information includes the previous frame of ego vehicle speed and the previous frame of heading angle change rate.

[0016] In an optional embodiment of the above step S2, obtaining the tracking equation of the lane line trajectory point of the previous frame by using the tracking coordinates of each lane line point in the lane line point set of the previous frame includes:

[0017] S23, performing cubic curve fitting on the lane line points in the lane line point set of the previous frame to obtain a fitted lane line point set;

[0018] S24. By using the fitted lane line point set, a least square method is used to minimize the objective function to obtain a lane line trajectory point tracking equation for the previous frame.

[0019] The embodiment of the present invention further provides a lane line trajectory point tracking system based on vehicle motion information, including a lane line point set acquisition module, a tracking equation acquisition module and a lane line tracking module.

[0020] The lane line point set acquisition module is used to perform lane line detection and coordinate system conversion on the collected front view image of the vehicle, and obtain a lane line point set in the vehicle coordinate system, wherein the lane line point set includes a lane line point set of a previous frame and a lane line point set of a current frame, and the lane line point set of the current frame is fitted to obtain a lane line detection equation of the current frame;

[0021] The tracking equation acquisition module is used for the tracking coordinate estimation module to estimate the tracking coordinates of each lane line point in the lane line point set of the previous frame according to the transformation relationship of the vehicle coordinate system of the previous and next frames, using the vehicle motion information of the previous frame, and obtain the lane line trajectory point tracking equation of the previous frame through the tracking coordinates of each lane line point in the lane line point set of the previous frame;

[0022] The lane line tracking module is used to fuse the lane line detection equation of the current frame and the lane line trajectory point tracking equation of the previous frame through a given fusion weight to obtain a lane line trajectory point tracking result.

[0023] In an optional embodiment of the above-mentioned lane line point set acquisition module, the lane line point set acquisition module includes a lane line detection module and a coordinate system conversion module, wherein the lane line detection module is used to use the front-view camera of the vehicle to collect the front view image, and use a pre-trained segmentation model or point regression model to perform lane line detection on the front view image to obtain a 2D image lane line point set; the coordinate system conversion module is used to perform coordinate system conversion on the 2D image lane line point set according to the posture conversion matrix between the pixel coordinate system and the vehicle coordinate system to obtain the lane line point set in the vehicle coordinate system.

[0024] In an optional embodiment of the above-mentioned tracking equation acquisition module, the tracking equation acquisition module includes a transformation relationship acquisition module, a coordinate estimation module and a tracking equation calculation module.

[0025] The transformation relationship acquisition module is used to establish the vehicle coordinate system in the subsequent frame, calculate the new position of a point in the vehicle coordinate system in the next frame according to the vehicle speed and the heading angle change rate, and obtain the transformation relationship between the vehicle coordinate systems in the previous and next frames;

[0026] The coordinate estimation module is used to estimate the tracking coordinates of each lane line point in the lane line point set of the previous frame by using the transformation relationship based on the vehicle speed and heading angle change rate of the previous frame of vehicle motion information collected;

[0027] The tracking equation calculation module is used to perform cubic curve fitting on the lane line points in the lane line point set of the previous frame to obtain a fitted lane line point set; through the fitted lane line point set, the least squares method is used to minimize the objective function to obtain the lane line trajectory point tracking equation of the previous frame.

[0028] An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any of the above-mentioned lane line trajectory point tracking methods based on vehicle motion information is implemented to solve the technical problem that existing lane line tracking methods are difficult to implement lane line tracking tasks in driving scenarios such as variable vehicle motion and variable road environment.

[0029] An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program for executing any of the above-mentioned lane line trajectory point tracking methods based on the vehicle motion information, so as to solve the technical problem that the existing lane line tracking methods are difficult to achieve lane line tracking tasks in driving scenarios such as changeable vehicle motion and changeable road environment.

[0030] Compared with the prior art, the beneficial effects that can be achieved by at least one of the above-mentioned technical solutions adopted in the embodiments of this specification include at least the following: the lane line trajectory point tracking method of the present invention estimates the lane line trajectory point tracking equation of the previous frame through the forward-looking image and motion information at the same time, and at the same time uses the estimated lane line trajectory point tracking equation of the previous frame and the current lane line detection equation for weighted fusion to obtain the tracking result. Compared with the traditional lane line tracking method based on the lane line position in the 2D image and based on the 3D lane line parameter filtering, the method of the present invention is not affected by the complex and changeable road environment and driving status. Only the vehicle motion information is used to explicitly express the lane line trajectory tracking point transformation relationship. At the same time, this method is more interpretable than the parameter filtering method, and is less affected by lane motion changes and road environment changes than the tracking method based on the image vehicle line position. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 A flow chart of a lane line trajectory point tracking method based on vehicle motion information disclosed in an embodiment of the present invention;

[0033] Figure 2 A schematic diagram of a lane line trajectory point tracking method based on vehicle motion information disclosed in an embodiment of the present invention;

[0034] Figure 3 The embodiment of the present invention discloses the position relationship between the lane line point in the current frame coordinate system and the vehicle coordinate system in the subsequent frame;

[0035] Figure 4 The architecture diagram of the lane line trajectory point tracking system based on vehicle motion information disclosed in an embodiment of the present invention;

[0036] Figure 5 A schematic diagram of a computer device disclosed in an embodiment of the present invention;

[0037] Among them, 401, lane line point set acquisition module; 402, tracking equation acquisition module; 403, lane line tracking module; 501, memory; 502, processor. DETAILED DESCRIPTION

[0038] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0039] The following describes the implementation methods of the present application through specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and the features of the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work belong to the scope of protection of the present application.

[0040] The embodiment of the present invention discloses a lane line trajectory point tracking method based on vehicle motion information, see Figure 1 and Figure 2 As shown, the method comprises the following steps:

[0041] S1. Perform lane line detection and coordinate system conversion on the collected front view image of the vehicle to obtain a lane line point set in the vehicle coordinate system, wherein the lane line point set includes a lane line point set of a previous frame and a lane line point set of a current frame, and perform fitting processing on the lane line point set of the current frame to obtain a lane line detection equation of the current frame;

[0042] S2, according to the transformation relationship between the ego-vehicle coordinate systems of the previous and next frames, using the ego-vehicle motion information of the previous frame to estimate the tracking coordinates of each lane line point in the lane line point set of the previous frame, and obtaining the lane line trajectory point tracking equation of the previous frame through the tracking coordinates of each lane line point in the lane line point set of the previous frame;

[0043] S3. The lane line detection equation of the current frame and the lane line trajectory point tracking equation of the previous frame are fused by a given fusion weight to obtain a lane line trajectory point tracking result.

[0044] In an optional embodiment of the above step S1, performing lane line detection and coordinate system conversion on the collected front view image of the vehicle to obtain a lane line point set in the vehicle coordinate system includes:

[0045] S11, using a calibrated front-view camera of the vehicle to collect the front-view image, and using a pre-trained segmentation model or point regression model to perform lane line detection on the front-view image to obtain a 2D image lane line point set;

[0046] S12. According to the pose conversion matrix between the pixel coordinate system and the vehicle coordinate system, the lane line point set of the 2D image is converted into a coordinate system to obtain a lane line point set in the vehicle coordinate system.

[0047] In an optional embodiment, the lane line detection equation of the current frame may be obtained by fitting the lane line point set of the current frame by minimizing the objective function using the least square method.

[0048] In an optional embodiment of the above step S11, the forward-looking camera is calibrated using a computer vision library or MATLAB camera calibration software to obtain a pose conversion matrix between the pixel coordinate system and the self-vehicle coordinate system, wherein the pose conversion matrix includes a rotation matrix and a translation vector of the forward-looking camera relative to the self-vehicle.

[0049] In an optional embodiment of the above step S2, estimating the tracking coordinates of each lane line point in the lane line point set of the previous frame by using the vehicle motion information of the previous frame according to the transformation relationship between the vehicle coordinate systems of the previous and subsequent frames includes:

[0050] S21, establishing the ego-vehicle coordinate system in the subsequent frame, calculating the new position of a certain point in the ego-vehicle coordinate system in the next frame ego-vehicle coordinate system according to the ego-vehicle speed (speed) and the heading angle change rate (yawrate), and obtaining the transformation relationship between the ego-vehicle coordinate systems in the previous and next frames;

[0051] S22, using the last frame of ego-vehicle motion information collected through the vehicle body CAN signal or the IMU sensor, and using the transformation relationship to estimate the tracking coordinates of each lane line point in the last lane line point set, wherein the last frame of ego-vehicle motion information includes the last frame of ego-vehicle speed and the last frame of heading angle change rate.

[0052] When the above step S21 is specifically implemented, refer to Figure 3 As shown in the figure, it is assumed that the coordinate system XOY is the current frame ego vehicle coordinate system, and F is a point in the current frame ego vehicle coordinate system, that is, the coordinates of point F in the XOY coordinate system are (x, y). At the next moment (i.e., at the later moment), when the vehicle moves from point O to point O', it can be considered that the vehicle has walked an arc in a short time. The radius of the arc R = V / yawrate, and the arc angle θ = yawrate*t, where V is the ego vehicle speed, yawrate is the rate of change of the heading angle, and t is the time it takes for the vehicle to move from point O to point O', that is, the frame interval time. At this time, the position deviation of the ego vehicle movement Δx = R*sin(θ), Δy = RR*cos(θ).

[0053] Establish an auxiliary coordinate system X'O'Y', assuming that X1O'Y1 is the coordinate system of the vehicle at the next moment. At this time, the position of a certain point F of the front vehicle in the coordinate system X1O'Y1 at the next moment is: X1 = O ′ CY1=O′D, combined with the coordinates of F in the XOY coordinate system (x, y), according to the geometric relationship, we can get:

[0054] AB=AO ′*cos(θ)=(y-Δy)*cos(θ);

[0055] AF = x - Δx;

[0056] AE=AF*sin(θ)=(x-Δx)*sin(θ);

[0057] O ′ B = (y-Δy)*sin(θ);

[0058] BC=FE=AF*cos(θ)=(x-Δx)*cos(θ);

[0059] According to the above geometric relationship, the coordinates of point F in the vehicle coordinate system at the next moment can be obtained as:

[0060] X1=O′C=O′B+BC=(y-Δy)*sin(θ)+(x-Δx)*cos(θ);

[0061] Y1=O'D=AB-AE=(y-Δy)*cos(θ)-(x-Δx)*sin(θ);

[0062] That is, the coordinates of a certain point of the vehicle in the new vehicle coordinate system can be summarized into the following six transformation relationships:

[0063] θ ego =yawrate*t;

[0064] X ego =Δx = R*sin(θ ego );

[0065] Y ego =Δy=RR*cos(θ ego );

[0066] X new =(x old -x ego )*cos(θ ego )+(Y old -Y ego )*sin(θ ego );

[0067] Y new =-(X old -X ego )*sin(θ ego )+(Y old -Y ego )*cos(θ ego );

[0068] Among them, θego is the heading angle; X ego Y is the lateral translation of the origin of the vehicle coordinate system; ego is the longitudinal translation of the origin of the vehicle coordinate system; (X old ,Y old ) is the coordinate of a point in the ego-vehicle coordinate system. The above formula can be used to obtain the coordinate position (X) of the point in the next frame tracked in the new ego-vehicle coordinate system. new ,Y new ).

[0069] In an optional embodiment of the above step S2, obtaining the tracking equation of the lane line trajectory point of the previous frame by using the tracking coordinates of each lane line point in the lane line point set of the previous frame includes:

[0070] S23, performing cubic curve fitting on the lane line points in the lane line point set of the previous frame to obtain a fitted lane line point set.

[0071] S24. By using the fitted lane line point set, a least square method is used to minimize the objective function to obtain a lane line trajectory point tracking equation for the previous frame.

[0072] When the above steps S23 and S24 are specifically implemented, the lane line point set in the previous frame of the vehicle coordinate system can be expressed as: After transforming each lane line point through the above six transformation relationships, the tracked point set can be expressed as: Using Cubic Curves The fitting lane line point set is obtained by fitting. The objective function is minimized by the least squares method. Solve to get the parameters To get the lane line trajectory point tracking equation of the previous frame. Similarly, the parameters in the lane line detection equation of the current frame can be obtained by minimizing the objective function using the least squares method.

[0073] When the above steps are implemented specifically, if there is a lane line in the front view image of the current frame, a fusion weight w can be given a value according to the actual usage scenario, and the lane line detection equation of the current frame and the lane line trajectory point tracking equation of the previous frame can be fused to output the final tracking result.

[0074] Specifically, the lane line trajectory point tracking equation of the previous frame can be expressed as The lane line detection equation for the current frame is expressed as The two equations are fused, and the fusion weight is set to w. Then the final lane line trajectory tracking equation after fusion is:

[0075]

[0076] in:

[0077]

[0078] The lane line trajectory point tracking method of the present invention estimates the lane line trajectory point tracking equation of the previous frame through the forward-looking image and motion information at the same time, and simultaneously uses the estimated lane line trajectory point tracking equation of the previous frame to perform weighted fusion with the current lane line detection equation to obtain the tracking result. Compared with the traditional lane line tracking method based on the lane line position in the 2D image and based on the 3D lane line parameter filtering, the method of the present invention is not affected by the complex and changeable road environment and driving status. Only the vehicle motion information is used to explicitly express the lane line trajectory tracking point transformation relationship. At the same time, compared with the parameter filtering method, this method is more interpretable and less affected by lane motion changes and road environment changes than the tracking method based on the image vehicle line position.

[0079] Based on the same inventive concept, a lane line trajectory point tracking system based on the vehicle motion information is also provided in an embodiment of the present invention, as described in the following embodiment. Since the principle of solving the problem by the lane line trajectory point tracking system is similar to the lane line trajectory point tracking method based on the vehicle motion information disclosed in the above embodiment, the implementation of the lane line trajectory point tracking system can refer to the implementation of the above lane line trajectory point tracking method, and the repeated parts will not be repeated. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements predetermined functions. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0080] Figure 4 is a structural block diagram of a lane line trajectory point tracking system based on vehicle motion information disclosed in an embodiment of the present invention, such as Figure 4 As shown, the lane line trajectory point tracking system includes a lane line point set acquisition module 401, a tracking equation acquisition module 402 and a lane line tracking module 403, and the structure is described below.

[0081] The lane line point set acquisition module 401 is used to perform lane line detection and coordinate system conversion on the collected front view image of the vehicle, and obtain a lane line point set in the vehicle coordinate system, wherein the lane line point set includes a lane line point set of a previous frame and a lane line point set of a current frame, and the lane line point set of the current frame is fitted to obtain a lane line detection equation of the current frame;

[0082] The tracking equation acquisition module 402 is used for the tracking coordinate estimation module to estimate the tracking coordinates of each lane line point in the lane line point set of the previous frame according to the transformation relationship of the vehicle coordinate system of the previous and next frames, using the vehicle motion information of the previous frame, and obtain the lane line trajectory point tracking equation of the previous frame through the tracking coordinates of each lane line point in the lane line point set of the previous frame;

[0083] The lane line tracking module 403 is used to fuse the lane line detection equation of the current frame and the lane line trajectory point tracking equation of the previous frame through a given fusion weight to obtain a lane line trajectory point tracking result.

[0084] In an optional embodiment of the above-mentioned lane line point set acquisition module 401, the lane line point set acquisition module 401 includes a lane line detection module and a coordinate system conversion module, wherein the lane line detection module is used to use the front-view camera of the vehicle to collect the front view image, and use a pre-trained segmentation model or point regression model to perform lane line detection on the front view image to obtain a 2D image lane line point set; the coordinate system conversion module is used to perform coordinate system conversion on the 2D image lane line point set according to the posture conversion matrix between the pixel coordinate system and the vehicle coordinate system to obtain the lane line point set in the vehicle coordinate system.

[0085] In an optional embodiment of the above-mentioned tracking equation acquisition module 402, the tracking equation acquisition module 402 includes a transformation relationship acquisition module, a coordinate estimation module and a tracking equation calculation module.

[0086] The transformation relationship acquisition module is used to establish the vehicle coordinate system in the subsequent frame, calculate the new position of a point in the vehicle coordinate system in the next frame according to the vehicle speed and the heading angle change rate, and obtain the transformation relationship between the vehicle coordinate systems in the previous and next frames;

[0087] The coordinate estimation module is used to estimate the tracking coordinates of each lane line point in the lane line point set of the previous frame by using the transformation relationship based on the vehicle speed of the previous frame of vehicle motion information collected and the heading angle change rate of the current frame;

[0088] The tracking equation calculation module is used to perform cubic curve fitting on the lane line points in the lane line point set of the previous frame to obtain a fitted lane line point set; through the fitted lane line point set, the least squares method is used to minimize the objective function to obtain the lane line trajectory point tracking equation of the previous frame.

[0089] In this embodiment, a computer device is provided, such as Figure 5 As shown, it includes a memory 501, a processor 502 and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any of the lane line trajectory point tracking methods based on the vehicle motion information is implemented.

[0090] Specifically, the computer device may be a computer terminal, a server or a similar computing device.

[0091] In this embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program for executing any of the above-mentioned lane line trajectory point tracking methods based on vehicle motion information.

[0092] Specifically, computer-readable storage media include permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include transitory media such as modulated data signals and carrier waves.

[0093] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

[0094] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A lane line trajectory point tracking method based on vehicle motion information, characterized in that: include: Perform lane line detection and coordinate system conversion on the collected front view image of the vehicle to obtain a lane line point set in the vehicle coordinate system, wherein the lane line point set includes a lane line point set of a previous frame and a lane line point set of a current frame, and perform fitting processing on the lane line point set of the current frame to obtain a lane line detection equation of the current frame; According to the transformation relationship between the ego-vehicle coordinate systems of the previous and next frames, the ego-vehicle motion information of the previous frame is used to estimate the tracking coordinates of each lane line point in the lane line point set of the previous frame, and the lane line trajectory point tracking equation of the previous frame is obtained through the tracking coordinates of each lane line point in the lane line point set of the previous frame; The lane line detection equation of the current frame and the lane line trajectory point tracking equation of the previous frame are fused through a given fusion weight to obtain a lane line trajectory point tracking result.

2. The lane line trajectory point tracking method based on vehicle motion information according to claim 1, characterized in that: The lane line detection and coordinate system conversion are performed on the collected front view image of the vehicle to obtain a lane line point set in the vehicle coordinate system, including: The front view image is collected by using a calibrated front view camera of the vehicle, and lane line detection is performed on the front view image by using a pre-trained segmentation model or a point regression model to obtain a 2D image lane line point set; According to the pose conversion matrix between the pixel coordinate system and the vehicle coordinate system, the 2D image lane line point set is converted into a coordinate system to obtain a lane line point set in the vehicle coordinate system.

3. The lane line trajectory point tracking method based on vehicle motion information according to claim 2, characterized in that: The forward-looking camera is calibrated using a computer vision library or MATLAB camera calibration software to obtain a posture conversion matrix between the pixel coordinate system and the vehicle coordinate system, wherein the posture conversion matrix includes a rotation matrix and a translation vector of the forward-looking camera relative to the vehicle.

4. The lane line trajectory point tracking method based on vehicle motion information according to claim 1, characterized in that: The method of estimating the tracking coordinates of each lane line point in the lane line point set of the previous frame by using the vehicle motion information of the previous frame according to the transformation relationship between the vehicle coordinate systems of the previous and subsequent frames includes: The ego-vehicle coordinate system in the subsequent frame is established, and the new position of a certain point in the ego-vehicle coordinate system in the next frame is calculated according to the ego-vehicle speed and the rate of change of the heading angle, and the transformation relationship between the ego-vehicle coordinate systems in the previous and next frames is obtained; The transformation relationship is used to estimate the tracking coordinates of each lane line point in the lane line point set by collecting the last frame of ego vehicle motion information, wherein the last frame of ego vehicle motion information includes the last frame of ego vehicle speed and the last frame of heading angle change rate.

5. The lane line trajectory point tracking method based on vehicle motion information according to claim 1 or 4, characterized in that: The obtaining of the tracking equation of the lane line trajectory point of the previous frame by the tracking coordinates of each lane line point in the lane line point set of the previous frame includes: Performing cubic curve fitting on the lane line points in the lane line point set of the previous frame to obtain a fitted lane line point set; The lane line point set is fitted and the least square method is used to minimize the objective function to obtain the lane line trajectory point tracking equation of the previous frame.

6. A lane line trajectory point tracking system based on vehicle motion information, characterized in that: include: A lane line point set acquisition module, which is used to perform lane line detection and coordinate system conversion on the collected front view image of the vehicle, to obtain a lane line point set in the vehicle coordinate system, wherein the lane line point set includes a lane line point set of a previous frame and a lane line point set of a current frame, and to perform fitting processing on the lane line point set of the current frame to obtain a lane line detection equation of the current frame; A tracking equation acquisition module, the tracking equation acquisition module is used to estimate the tracking coordinates of each lane line point in the lane line point set of the previous frame according to the transformation relationship of the vehicle coordinate system of the previous and next frames, using the vehicle motion information of the previous frame, and obtain the lane line trajectory point tracking equation of the previous frame through the tracking coordinates of each lane line point in the lane line point set of the previous frame; The lane line tracking module is used to fuse the lane line detection equation of the current frame and the lane line trajectory point tracking equation of the previous frame through a given fusion weight to obtain a lane line trajectory point tracking result.

7. The lane line trajectory point tracking system based on vehicle motion information according to claim 6, characterized in that: The lane line point set acquisition module includes a lane line detection module and a coordinate system conversion module. The lane line detection module is used to use the front-view camera of the vehicle to collect the front view image, and use a pre-trained segmentation model or point regression model to perform lane line detection on the front view image to obtain a 2D image lane line point set; the coordinate system conversion module is used to perform coordinate system conversion on the 2D image lane line point set according to the posture conversion matrix between the pixel coordinate system and the vehicle coordinate system to obtain the lane line point set in the vehicle coordinate system.

8. The lane line trajectory point tracking system based on vehicle motion information according to claim 6, characterized in that: The tracking equation acquisition module includes a transformation relationship acquisition module, a coordinate estimation module and a tracking equation calculation module; The transformation relationship acquisition module is used to establish the vehicle coordinate system in the subsequent frame, calculate the new position of a point in the vehicle coordinate system in the next frame according to the vehicle speed and the heading angle change rate, and obtain the transformation relationship of the vehicle coordinate system in the previous and next frames; The coordinate estimation module is used to estimate the tracking coordinates of each lane line point in the lane line point set of the previous frame by using the transformation relationship based on the vehicle speed and heading angle change rate of the previous frame of vehicle motion information collected; The tracking equation calculation module is used to perform cubic curve fitting on the lane line points in the previous lane line point set to obtain a fitted lane line point set; through the fitted lane line point set, the least squares method is used to minimize the objective function to obtain the lane line trajectory point tracking equation of the previous frame.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the lane line trajectory point tracking method based on the vehicle motion information described in any one of claims 1 to 5 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the lane line trajectory point tracking method based on vehicle motion information according to any one of claims 1 to 5.

Images