A method, device, equipment and medium for detecting illegal lane-changing behavior

Abstract

Embodiments of the present application provide a method, device, equipment and medium for detecting illegal lane changing behavior, which comprises: collecting lane line point sets of a target lane line, and performing lane line fitting on the collected lane line point sets by using a quadratic curve equation to obtain a curve function expression of the target lane line; collecting multiple frames of images of a vehicle driving on the target lane line in real time, identifying the driving positions of the vehicle in each frame of image to obtain the driving positions of the target vehicle in each frame of image; determining the positional relationship between the vehicle position corresponding to each frame of image and the position of the fitted target lane line by using the parameters in the curve function expression and the driving positions of the target vehicle in each frame of image; and determining that the target vehicle has committed illegal lane changing behavior when the target vehicle is located in different lanes in different frames of image according to the positional relationship corresponding to the multiple frames of image. The above method reduces the calculation amount for judging illegal lane changing behavior and is suitable for scenes with large turning angles.

Description

Technical Field

[0001] This application relates to the field of traffic violation detection technology, and in particular to a method, device, equipment and medium for detecting illegal lane changing behavior. Background Technology

[0002] Illegal lane changing generally refers to changing lanes over solid lines, that is, changing lanes while the lane lines on the road are solid. In actual road monitoring, lane changing is usually captured on vertical lanes because the lane lines on vertical lanes are straight lines. Therefore, traditional radar detection methods or current intelligent image recognition can easily identify the lane line positions, ensuring the accuracy of the capture.

[0003] Because curves present challenges to drivers' field of vision and visibility compared to straight roads, illegal lane changes can lead to more serious consequences. Therefore, addressing illegal lane changes on curves is a crucial issue in current intelligent transportation projects.

[0004] Currently, the methods for detecting illegal lane changes on curves typically involve fitting lane lines for specific scenarios and then judging illegal lane changes based on the fitted lane lines. This method is only applicable to specific scenarios and cannot be applied to all curves. It cannot support scenarios with large turning angles, and the process of judging illegal lane changes is relatively complex, resulting in inaccurate detection results. Summary of the Invention

[0005] This invention provides a method, apparatus, device, and medium for detecting illegal lane change behavior, which solves the problems that existing technologies for detecting illegal lane change behavior cannot support scenarios with large turning angles, and that the judgment process is relatively complex and the detection results are not accurate enough.

[0006] A first aspect of this application provides a method for detecting illegal lane changing behavior, including:

[0007] Collect the lane line point set of the target lane line, and use the quadratic curve equation to fit the lane line based on the collected lane line point set to obtain the curve function expression of the target lane line.

[0008] The system acquires multiple frames of images of vehicles traveling on the target lane in real time, identifies the vehicle's position in each frame, and obtains the position of the target vehicle in each frame.

[0009] By using the parameters in the curve function expression and the driving position of the target vehicle in each frame image, the positional relationship between the vehicle position in each frame image and the fitted target lane line is determined.

[0010] Based on the positional relationship between multiple frames of images, it is determined that the target vehicle has committed an illegal lane change when it is located in different lanes in different frames of images.

[0011] The above method solves the problems of difficulty and error in capturing images on curves by traditional methods. At the same time, it uses the parameters in the curve function expression and the driving position of the target vehicle in each frame image to determine the positional relationship between the vehicle position in each frame image and the fitted target lane line, which greatly saves the amount of computation. In addition, this method can adapt to curves of various angles and has a wider range of applications.

[0012] An optional implementation involves using the parameters in the curve function expression and the driving position of the target vehicle in each frame of the image to determine the positional relationship between the vehicle position corresponding to each frame of the image and the fitted target lane line, including:

[0013] For each frame of image, the positional relationship between the vehicle position in the image and the fitted target lane line is determined based on the value of det(L)*F(x,y) and the magnitude of zero.

[0014] in, F(x,y)=0 is the curve function expression of the target lane line, A′, B′, C′, D′, E′ are the parameters in the curve function expression, and (x,y) are the coordinates of the points on the target vehicle in each frame of the image.

[0015] An optional implementation involves determining the positional relationship between the vehicle position corresponding to each frame of the image and the fitted target lane line, including:

[0016] Extract the bottom envelope of the target vehicle in each frame image and determine the center point of the bottom envelope;

[0017] By using the parameters in the curve function expression, the positional relationship between the center point of the bottom envelope and the fitted target lane line is determined.

[0018] One optional implementation involves collecting a set of lane line points for the target lane line, including:

[0019] The target lane line image is acquired and input into the semantic segmentation model. The semantic segmentation model is then used to identify the type of the target lane line and the location information of each point in the lane line point set.

[0020] When the target lane line type is determined to be a solid line, the position information of each point in the lane line point set of the target lane line output by the semantic segmentation model is obtained.

[0021] An optional implementation method, when there are multiple target lane lines, after determining the positional relationship between the vehicle position corresponding to each frame image and the fitted target lane line, further includes:

[0022] Based on the arrangement order of multiple target lane lines and the positional relationship between the vehicle position in each frame and the fitted target lane line, the lane in which the target vehicle is located in each frame is determined.

[0023] An optional implementation involves, after determining the positional relationship between the vehicle position in each frame and the fitted target lane line using the parameters in the curve function expression and the target vehicle's driving position in each frame image, the implementation further includes:

[0024] Based on the positional relationship of each frame image, it is determined that the target vehicle has crossed the lane when it occupies a different lane in any frame image.

[0025] An optional implementation involves determining the positional relationship between the vehicle position corresponding to each frame of the image and the fitted target lane line, including:

[0026] Extract the envelope regions of different headlights / tires of the target vehicle in each frame image, and determine the center point of each envelope region;

[0027] By using the parameters in the curve function expression, the positional relationship between the center point of each envelope region in each frame of the image and the fitted target lane line is determined;

[0028] Determining whether the target vehicle is located in a different lane in any frame of the image includes:

[0029] When the center point of different envelope regions of the target vehicle is located on different sides of the fitted target lane line, it is determined that the target vehicle occupies different lanes.

[0030] A second aspect of this application provides an illegal lane change detection device, comprising:

[0031] The fitting module is used to collect the lane line point set of the target lane line, and to fit the lane line using a quadratic curve equation based on the collected lane line point set to obtain the curve function expression of the target lane line.

[0032] The acquisition module is used to acquire multiple frames of images of vehicles traveling on the target lane in real time, identify the driving position of the vehicle in each frame of the image, and obtain the driving position of the target vehicle in each frame of the image.

[0033] The positional relationship determination module is used to determine the positional relationship between the vehicle position in each frame image and the fitted target lane line by using the parameters in the curve function expression and the driving position of the target vehicle in each frame image.

[0034] The illegal lane change determination module is used to determine whether a target vehicle has committed an illegal lane change behavior when it is located in different lanes in different frames of images, based on the positional relationship between the corresponding images.

[0035] A third aspect of this application provides an illegal lane change detection device, comprising:

[0036] Collect the lane line point set of the target lane line, and use the quadratic curve equation to fit the lane line based on the collected lane line point set to obtain the curve function expression of the target lane line.

[0037] The system acquires multiple frames of images of vehicles traveling on the target lane in real time, identifies the vehicle's position in each frame, and obtains the position of the target vehicle in each frame.

[0038] By using the parameters in the curve function expression and the driving position of the target vehicle in each frame image, the positional relationship between the vehicle position in each frame image and the fitted target lane line is determined.

[0039] Based on the positional relationship between multiple frames of images, it is determined that the target vehicle has committed an illegal lane change when it is located in different lanes in different frames of images.

[0040] A fourth aspect of this application provides a computer storage medium, including: computer program instructions, which, when run on a computer, cause the computer to perform any step in the above-described illegal lane change detection method.

[0041] In addition, the technical effects of any of the implementation methods in the second to fourth aspects mentioned above can be found in the technical effects of different implementation methods in the above-mentioned illegal lane change behavior detection method, which will not be repeated here. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This application provides an overall flowchart of a process for detecting illegal lane changing behavior.

[0044] Figure 2 A flowchart illustrating a method for detecting illegal lane changing behavior provided in an embodiment of this application;

[0045] Figure 3 A schematic diagram of a single-lane road provided in an embodiment of this application;

[0046] Figure 4 A schematic diagram of a multi-lane road provided in an embodiment of this application;

[0047] Figure 5 A schematic diagram illustrating an illegal lane-changing behavior of a vehicle, provided as an embodiment of this application;

[0048] Figure 6 A schematic diagram illustrating a vehicle crossing a lane line, provided as an embodiment of this application;

[0049] Figure 7 A schematic diagram of an illegal lane change detection device provided in an embodiment of this application;

[0050] Figure 8 This is a schematic diagram of an illegal lane change detection device provided in an embodiment of this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will now be described in further detail with reference to the accompanying drawings.

[0052] The application scenarios described in this application are for the purpose of more clearly illustrating the technical solutions of this application, and do not constitute a limitation on the technical solutions provided in this application. Those skilled in the art will understand that with the emergence of new application scenarios, the technical solutions provided in this application are also applicable to similar technical problems. In the description of this application, unless otherwise stated, "multiple" means two or more.

[0053] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application.

[0054] Illegal lane changing generally refers to changing lanes over solid lines, that is, changing lanes while the lane lines are solid. In actual road monitoring, lane change cameras are mostly used to capture violations on vertical lanes. This is because lane lines are straight lines on vertical lanes, making them easy to identify using traditional radar detection methods or modern intelligent image recognition, resulting in high accuracy.

[0055] Compared to straight roads, curves can lead to more serious consequences when illegal lane changes occur due to factors such as the driver's field of vision and sight distance. Therefore, solving the problem of illegal lane changes and lane crossing detection on curves is an important issue in current intelligent transportation projects. Among the most important issues are how to determine lane lines and how to calculate the relative position of vehicles and lane lines.

[0056] Currently, there are two main methods for detecting illegal lane changes on curves. One method involves extracting a set of lane line points, performing piecewise fitting, and then using the fitted lane line to determine the vehicle's illegal driving behavior. This method requires a large number of lane line points, resulting in significant computational overhead. Furthermore, its applicability is limited, as it cannot be applied to scenarios with large curves, and the determination process for illegal lane changes is complex, leading to inaccurate detection results. The other method involves extracting key points on the lane lines and fitting the lane lines using a cubic function based on these key points. However, this method is prone to confusion between key points in different lanes during key point collection, resulting in poor fitting performance. It is also only applicable to specific scenarios and cannot support scenarios with large turning angles.

[0057] Based on the above problems, this application provides a method, apparatus, device and medium for detecting illegal lane change behavior, which solves the problems that the above-mentioned illegal lane change behavior detection cannot support scenarios with large turning angles, and that the judgment process is relatively complicated and the detection results are not accurate enough.

[0058] Figure 1 This application provides an overall flowchart of a process for detecting illegal lane changes, as illustrated in the embodiments of this application. Figure 1 As shown, image data is first acquired using an image acquisition device. The lane line point set is obtained from the image data, and the lane line is fitted using a quadratic curve equation. Then, the position information of the target vehicle is obtained from the image data, and the image data of the target vehicle is acquired in real time to track the vehicle's position in real time. Finally, based on the curve function expression of the fitted target lane line, it is detected whether the target vehicle has illegally changed lanes or crossed the line.

[0059] Figure 2 This is a flowchart illustrating a method for detecting illegal lane changing behavior provided in an embodiment of this application; as shown below. Figure 2 As shown in the figure, this application provides a method for detecting illegal lane changing behavior, including:

[0060] Step 201: Collect the lane line point set of the target lane line, and use the quadratic curve equation to fit the lane line based on the collected lane line point set to obtain the curve function expression of the target lane line.

[0061] In some embodiments, when the target lane line is not obscured, an image of the target lane line is acquired in advance by an image acquisition device (such as a camera, image sensor, etc.), and a pre-trained semantic segmentation model is used to extract the type of the target lane line and the position information of each point in the point set. The type of lane line includes solid lines and dashed lines, and the position information of each point can be the position coordinates of each point.

[0062] As an optional implementation method, the collection of lane line point sets for the target lane line includes:

[0063] The target lane line image is acquired and input into the semantic segmentation model. The semantic segmentation model is then used to identify the type of the target lane line and the location information of each point in the lane line point set.

[0064] When the target lane line type is determined to be a solid line, the position information of each point in the lane line point set of the target lane line output by the semantic segmentation model is obtained.

[0065] The above method uses a semantic segmentation model to identify the type of target lane lines and the location information of each point in the lane line point set, saving manual configuration time and eliminating the need for maintenance when the equipment is moved due to external factors, thus saving labor costs.

[0066] In some embodiments, a set of lane line points for the target lane line can be periodically collected at preset intervals, and the target lane line can be fitted to periodically update the curve function expression of the previously fitted target lane line, thereby ensuring the accuracy of the target lane line fitting results.

[0067] Step 202: Real-time acquisition of multiple frames of images of vehicles traveling on the target lane, vehicle driving position recognition of each frame of images, and obtaining the driving position of the target vehicle in each frame of images;

[0068] In some embodiments, after acquiring multiple frames of images of vehicles traveling on the target lane in real time, each frame of the image is decoded, and the vehicle driving position is identified in each frame of the image using a vehicle detection model obtained through deep learning. The position information of the target vehicle in each frame of the image is obtained, which includes the position coordinates of each point on the target vehicle. At the same time, the vehicle detection model can also be used to identify the position information of different components of the target vehicle, such as headlights and tires, in each frame of the image.

[0069] As an optional implementation, embodiments of this application can also utilize a license plate recognition model to perform license plate recognition of the target vehicle based on multiple frames of images of vehicles traveling on the target lane in real time, thereby obtaining the license plate number of the target vehicle.

[0070] Step 203: Using the parameters in the curve function expression and the driving position of the target vehicle in each frame image, determine the positional relationship between the vehicle position corresponding to each frame image and the fitted target lane line.

[0071] After determining the curve function expression of the fitted target lane line, the positional relationship between the points on the target vehicle and the fitted target lane line in each frame image can be determined based on the obtained target vehicle position information and the discriminant of the point and curve position in the quadratic curve equation.

[0072] Step 204: Based on the positional relationship between the multiple frames of images, determine when the target vehicle is located in different lanes in different frames of images, and determine that the target vehicle has committed an illegal lane change.

[0073] In some embodiments, the positional relationship between the center point of the target vehicle in each frame image (which can be selected as the center point of its bottom to improve the accuracy of the determination) and the target lane line can be judged to determine the lane in which the target vehicle is located in different frame images. By combining multiple frame images, it can be determined that the target vehicle has committed an illegal lane change when it is located in different lanes in different frame images.

[0074] The above method uses a quadratic curve equation to fit the target lane line. Based on the parameters in the curve function expression of the fitted target lane line and the driving position of the target vehicle in each frame of the image, the positional relationship between the target vehicle and the target lane line can be determined simply and quickly, thereby determining whether the target vehicle has committed an illegal lane change. At the same time, the method in this embodiment can adapt to more scenarios than traditional methods, such as being applicable to curves with various directions and curvatures.

[0075] The following section elaborates on the process in step 201 above, which involves fitting lane lines using a quadratic curve equation based on the collected lane line point set to obtain the curve function expression of the target lane line.

[0076] As an optional implementation, before fitting the lane lines using a quadratic curve equation based on the collected lane line point set, the following steps are also included:

[0077] The system samples points from the lane line point set, collecting the center point of the lane line point set along the width direction of the target lane line, and uniformly sampling along the length direction of the target lane line at preset intervals. That is, only one point is retained in the same width direction of the lane line, while sampling is performed at fixed intervals along the length direction. These fixed intervals can be set according to actual needs. In specific scenarios, uniform sampling may not be performed; instead, points from the lane line point set may be sampled according to preset rules.

[0078] The process of using the quadratic curve equation for lane line fitting is explained in detail below.

[0079] (1) The derivation of the equation of the quadratic curve is as follows:

[0080] The implicit equation in standard form of a conic section is as follows:

[0081] Ax 2 +By 2 +Cxy+Dx+Ey+F=0 (1)

[0082] When C 2 When -4AB < 0, the shape of the above quadratic curve equation is an ellipse;

[0083] When C 2 When -4AB = 0, the shape of the above quadratic curve equation is a parabola;

[0084] When C 2 When -4AB>0, the shape of the above quadratic curve equation is a hyperbola.

[0085] Dividing equation (1) by -F, we obtain a new equation with a constant term of -1 as follows:

[0086]

[0087] By replacing each parameter in equation (2) with the new parameters, we can obtain the following equation:

[0088] A′x 2 +B′y 2 +C′xy+D′x+E′y-1=0 (3)

[0089] When C ′2 -4A ′ B ′ When <0, the shape of the above equation (3) is an ellipse;

[0090] When C ′2 -4A ′ B ′ When =0, the shape of the above equation (3) is a parabola;

[0091] When C ′2 -4A ′ B ′ When >0, the shape of the above equation (3) is a hyperbola.

[0092] Representing equation (3) in matrix form, we obtain the following equation:

[0093]

[0094] The derivation using the matrix form of the least squares method is as follows:

[0095] The matrix form of a set of linear equations is:

[0096]

[0097] The solution for X is:

[0098]

[0099] Among them (A) T A) -1 A T Let A be the generalized inverse of matrix A. T It is the transpose of matrix A.

[0100] (2) The process of fitting lane lines using the quadratic curve equation based on the lane line point set is as follows:

[0101] Assume the collected lane line point set includes n points, whose position coordinates are: {(x1,y1),(x2,y2)…(x... n ,y n )}.

[0102] Substituting the coordinates of the above n points into equation (4), we obtain the following equation (7):

[0103]

[0104] From equations (5) and (7) above, we can obtain:

[0105]

[0106] The parameter X can be obtained by equation (6).

[0107] make Then we can obtain the matrix form of the quadratic curve equation based on equations (3) and (4), as follows:

[0108]

[0109] make:

[0110]

[0111] (3) When there are multiple target lane lines, the target lane lines are fitted as follows:

[0112] After collecting the lane line point set of the target lane line, the connected component algorithm is used to determine the connected regions in the point set, and the set of points in each connected region is taken as the lane line point set of the same lane line.

[0113] Suppose there are n lane lines on the road. According to the above method, based on the set of lane line points corresponding to each target lane line, the target lane line is fitted, and the equation of the fitted lane line is as follows:

[0114] F n (x,y) = 0 (12)

[0115] The following elaborates in detail the process of determining the positional relationship between the vehicle position corresponding to each frame image and the fitted target lane line by using the parameters in the curve function expression and the driving position of the target vehicle in each frame image in step 203 above.

[0116] First, the following conclusions can be obtained according to the properties of the quadratic curve equation:

[0117] When the determinant det(L) of the matrix L > 0, the points where F(x,y) > 0 are inside the quadratic curve, the points where F(x,y) < 0 are outside the quadratic curve, and the points where F(x,y) = 0 are on the quadratic curve.

[0118] When the determinant det(L) of the matrix L < 0, the points where F(x,y) < 0 are inside the quadratic curve, the points where F(x,y) > 0 are outside the quadratic curve, and the points where F(x,y) = 0 are on the quadratic curve.

[0119] Furthermore, the following conclusions can be obtained: the points where det(L)*F(x,y) > 0 are inside the quadratic curve, the points where det(L)*F(x,y) < 0 are outside the quadratic curve, and the points where det(L)*F(x,y) = 0 are on the quadratic curve.

[0120] Furthermore, when there are multiple (n) lane lines and the lanes are lanes [1,..., n + 1] starting from the innermost side, the following conclusions also exist:

[0121] When i = 1 and det(L i )*F i (x,y) > 0, this point is inside lane line 1, that is, on lane line 1;

[0122] When 1 ≤ i < n and det(L i )*F i (x,y) < 0, det(L i+1 )*F i+1 (x,y) > 0, this point is outside lane line i and inside lane line i + 1, that is, on lane line i + 1;

[0123] When i = n and det(L i )*F i (x,y) < 0, this point is outside lane line n, that is, on lane line n + 1.

[0124] As an optional implementation, the positional relationship between the vehicle position in each frame and the fitted target lane line is determined using the parameters in the curve function expression and the driving position of the target vehicle in each frame image, including:

[0125] For each frame of image, the positional relationship between the vehicle position corresponding to the image and the fitted target lane line is determined based on the value of det(L)*F(x,y) and the magnitude of zero;

[0126] in, F(x,y)=0 is the curve function expression of the target lane line, A′, B′, C′, D′, E′ are the parameters in the curve function expression, and (x,y) are the coordinates of the points on the target vehicle in each frame of the image.

[0127] In practice, the point set of the target vehicle in each frame of the image is obtained, and the points in the point set are substituted into the det(L)*F(x,y) formula. By comparing the result with the magnitude of zero, the positional relationship between the point and the target lane line (whether it is inside or outside the lane line) can be determined. In this way, the positional relationship between the vehicle position corresponding to each frame of the image and the fitted target lane line can be determined.

[0128] As an optional implementation, when there are multiple target lane lines, after determining the positional relationship between the vehicle position corresponding to each frame image and the fitted target lane line, the method further includes:

[0129] Based on the arrangement order of multiple target lane lines and the positional relationship between the vehicle position in each frame and the fitted target lane line, the lane in which the target vehicle is located in each frame is determined.

[0130] The following combination Figure 3 and Figure 4 Give an example to illustrate the process of determining the above positional relationships.

[0131] Figure 3 This is a schematic diagram of a single lane line provided in an embodiment of this application, wherein label 1 represents a lane line, and its fitted curve function expression is F1(x,y)=0, with lane 1 on the inner side and lane 2 on the outer side.

[0132] like Figure 3 As shown, for each frame of the image, after obtaining the position of the target vehicle in the image, that is, the coordinates of the target vehicle point set, the coordinates of the point in the target vehicle point set are substituted into det(L)*F(x,y). If the value of det(L)*F(x,y) is greater than 0, it means that the point is located in lane 1. If the value of det(L)*F(x,y) is less than 0, it means that the point is located in lane 2.

[0133] Figure 4 This is a schematic diagram of a multi-lane line provided in an embodiment of this application. Label 1 represents lane line 1, whose fitted curve function expression is F1(x,y)=0, with lane 1 inside and lane 2 outside; label 2 represents lane line 2, whose fitted curve function expression is F2(x,y)=0, with lane 2 inside and lane 3 outside; label 3 represents lane line 3, whose fitted curve function expression is F3(x,y)=0, with lane 3 inside and lane 4 outside.

[0134] like Figure 4 As shown, for each frame of the image, after obtaining the position of the target vehicle in the image, i.e., the coordinates of the target vehicle point set, the coordinates of the points in the target vehicle point set are sequentially substituted into det(L). i )*F i In (x,y), it should be noted that, in this embodiment, the coordinates of this point are not substituted into the determination equation det(L) corresponding to each lane line. i )*F i Instead of using (x,y), the calculation proceeds sequentially from the innermost to the outermost, or from the outermost to the innermost, and stops after determining the lane line where the point is located.

[0135] Specifically, the above process is described below in order from lane line 1 to lane line n:

[0136] If the value of det(L1)*F1(x,y) is greater than 0, it means that the point is located inside lane line 1, that is, on lane 1, and the positional relationship between the point and other lane lines (lane line 2 and lane line 3) is no longer determined.

[0137] If the value of det(L1)*F1(x,y) is less than 0 and the value of det(L2)*F2(x,y) is greater than 0, it means that the point is located outside lane line 1 and inside lane line 2, that is, on lane 2, and the positional relationship between the point and lane line 3 is no longer determined.

[0138] If the value of det(L2)*F2(x,y) is less than 0 and the value of det(L3)*F3(x,y) is greater than 0, it means that the point is located outside lane line 2 and inside lane line 3, that is, on lane 3.

[0139] If the value of det(L3)*F3(x,y) is less than 0, it means that the point is located outside lane line 3, i.e., on lane 4.

[0140] As an optional implementation, step 203 above, determining the positional relationship between the vehicle position corresponding to each frame image and the fitted target lane line, includes:

[0141] Extract the bottom envelope of the target vehicle in each frame image and determine the center point of the bottom envelope;

[0142] Using the parameters in the curve function expression, the positional relationship between the center point of the bottom envelope and the fitted target lane line is determined.

[0143] like Figure 5 As shown, in step 202 above, after the target vehicle position is detected using the vehicle detection model, the bottom envelope of the target vehicle is extracted, such as the rectangular envelope area at the bottom of the target vehicle, and the position information (coordinates) of the center point of the envelope area is determined. Using the above method, the positional relationship between the center point and the target lane line is determined, and the lane in which the center point is located is further determined and identified as the lane in which the target vehicle is located.

[0144] When it is determined that the target vehicle is located in different lanes in each frame, it can be determined that the target vehicle has committed an illegal lane change. After determining that the target vehicle has committed an illegal lane change, the license plate number of the target vehicle and the lane change information can also be determined and recorded.

[0145] The following is Figure 3 Using the single-lane line shown as an example, the process for determining the above-mentioned illegal lane change is described.

[0146] For each frame of the image, the coordinates of the center point are substituted into the formula det(L)*F(x,y). If det(L)*F(x,y)>0, it is determined that the center point is located inside the lane line, and the target vehicle is considered to be located in lane 1. If det(L)*F(x,y)<0, it is determined that the center point is located inside the lane line, and the target vehicle is considered to be located in lane 2.

[0147] After determining the lane of the target vehicle in each frame of the image, and based on multiple frames of images, if the target vehicle is located in different lanes in each frame, it can be determined that the target vehicle has committed an illegal lane change.

[0148] Specifically, if the discriminant changes from det(L)*F(x,y)<0 to det(L)*F(x,y)>0, then it can be determined that the target vehicle has changed from lane 2 to lane 1.

[0149] If the discriminant changes from det(L)*F(x,y)>0 to det(L)*F(x,y)<0, then it can be determined that the target vehicle changes from lane 1 to lane 2.

[0150] As an optional implementation, after determining the positional relationship between the vehicle position in each frame and the fitted target lane line using the parameters in the curve function expression and the driving position of the target vehicle in each frame image in step 204, the method further includes:

[0151] Based on the positional relationship of each frame image, it is determined that the target vehicle has committed lane-crossing behavior when it occupies different lanes in any frame image.

[0152] As an optional implementation, determining the positional relationship between the vehicle position corresponding to each frame image and the fitted target lane line includes:

[0153] Extract the envelope regions of different headlights / tires of the target vehicle in each frame image, and determine the center point of each envelope region;

[0154] Using the parameters in the curve function expression, the positional relationship between the center point of each envelope region in each frame of the image and the fitted target lane line is determined;

[0155] Determining whether the target vehicle is located in a different lane in any frame of the image includes:

[0156] When the center point of different envelope regions of the target vehicle is located on different sides of the fitted target lane line, it is determined that the target vehicle occupies different lanes.

[0157] The following describes the process of determining whether the target vehicle has crossed the line, using vehicle lights as an example.

[0158] like Figure 6 As shown, in step 202 above, when using the vehicle detection model to detect the target vehicle's position, the position of the target vehicle's headlights can also be detected. The envelope regions (rectangular detection boxes) of the two headlights of the target vehicle in each frame are extracted, and the coordinates (x, y, y) of the center point of each envelope region are determined. l ,y l ),(x r ,y r ).

[0159] Substitute the coordinates of the two center points into the equation det(L)*F(x,y) to determine the positional relationship between the two center points and the fitted target lane line. When the two center points are located on different sides of the target lane line, it can be determined that the target vehicle occupies different lanes at the same time, that is, the target vehicle has crossed the line.

[0160] Since the value of det(L) depends only on the curve function expression of the target lane line and is independent of the coordinates of the center point, it can be determined by judging F(x) l ,y l )*F(x r ,y r If the value is less than 0, it indicates that the two center points are located on different sides of the target lane line.

[0161] The above method allows for the determination of headlight / tire position detection and vehicle detection through a vehicle detection model, thus providing information such as headlight position without additional computational load.

[0162] Based on the same disclosed concept, this application also provides an illegal lane change detection device. Since this device is the same as the device in the method of this application, and the principle of the device in solving the problem is similar to that of the method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0163] Figure 7 Please refer to the schematic diagram of an illegal lane change detection device provided in this application embodiment. Figure 7 This application provides a device for detecting illegal lane changing behavior, the device comprising:

[0164] The fitting module 701 is used to collect the lane line point set of the target lane line, and to perform lane line fitting using the quadratic curve equation based on the collected lane line point set to obtain the curve function expression of the target lane line.

[0165] The acquisition module 702 is used to acquire multiple frames of images of vehicles traveling on the target lane in real time, identify the driving position of each frame of images, and obtain the driving position of the target vehicle in each frame of images.

[0166] The position relationship determination module 703 is used to determine the position relationship between the vehicle position in each frame image and the fitted target lane line by using the parameters in the curve function expression and the driving position of the target vehicle in each frame image.

[0167] The illegal lane change determination module 704 is used to determine whether the target vehicle has committed an illegal lane change behavior when it is located in different lanes in different frames of images based on the positional relationship between the corresponding images.

[0168] Optionally, the aforementioned positional relationship determination module 703 is used to determine the positional relationship between the vehicle position corresponding to each frame image and the fitted target lane line, based on the parameters in the curve function expression and the driving position of the target vehicle in each frame image, including:

[0169] For each frame of image, the positional relationship between the vehicle position in the image and the fitted target lane line is determined based on the value of det(L)*F(x,y) and the magnitude of zero.

[0170] in, F(x,y)=0 is the curve function expression of the target lane line, A′, B′, C′, D′, E′ are the parameters in the curve function expression, and (x,y) are the coordinates of the points on the target vehicle in each frame of the image.

[0171] Optionally, the positional relationship determination module 703 is used to determine the positional relationship between the vehicle position corresponding to each frame image and the fitted target lane line, including:

[0172] Extract the bottom envelope of the target vehicle in each frame image and determine the center point of the bottom envelope;

[0173] By using the parameters in the curve function expression, the positional relationship between the center point of the bottom envelope and the fitted target lane line is determined.

[0174] Optionally, the above-mentioned fitting module 701 is used to collect the lane line point set of the target lane line, including:

[0175] The target lane line image is acquired and input into the semantic segmentation model. The semantic segmentation model is then used to identify the type of the target lane line and the location information of each point in the lane line point set.

[0176] When the target lane line type is determined to be a solid line, the position information of each point in the lane line point set of the target lane line output by the semantic segmentation model is obtained.

[0177] Optionally, when there are multiple target lane lines, after the positional relationship determination module 703 determines the positional relationship between the vehicle position corresponding to each frame image and the fitted target lane line, it further includes:

[0178] Based on the arrangement order of multiple target lane lines and the positional relationship between the vehicle position in each frame and the fitted target lane line, the lane in which the target vehicle is located in each frame is determined.

[0179] Optionally, the aforementioned positional relationship determination module 703, after determining the positional relationship between the vehicle position corresponding to each frame image and the fitted target lane line using the parameters in the curve function expression and the driving position of the target vehicle in each frame image, further includes:

[0180] Based on the positional relationship of each frame image, it is determined that the target vehicle has crossed the lane when it occupies a different lane in any frame image.

[0181] Optionally, the aforementioned positional relationship determination module 703 is used to determine the positional relationship between the vehicle position corresponding to each frame of the image and the fitted target lane line, including:

[0182] Extract the envelope regions of different headlights / tires of the target vehicle in each frame image, and determine the center point of each envelope region;

[0183] By using the parameters in the curve function expression, the positional relationship between the center point of each envelope region in each frame of the image and the fitted target lane line is determined;

[0184] Determining whether the target vehicle is located in a different lane in any frame of the image includes:

[0185] When the center point of different envelope regions of the target vehicle is located on different sides of the fitted target lane line, it is determined that the target vehicle occupies different lanes.

[0186] Based on the same disclosed concept, this application also provides an illegal lane change detection device. Since this device is the same as the device in the method of this application, and the principle of the device in solving the problem is similar to that of the method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0187] Those skilled in the art will understand that various aspects of this application can be implemented as a system, method, or program product. Therefore, various aspects of this application can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, collectively referred to herein as a "circuit," "module," or "system."

[0188] In some possible implementations, the device according to this application may include at least one processor and at least one memory. The memory stores program code that, when executed by the processor, causes the processor to perform the steps in the illegal lane change detection method according to various exemplary embodiments of this application described above.

[0189] The following reference Figure 8 To describe the device 800 according to this embodiment of the present application. Figure 8 The device 800 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0190] like Figure 8 As shown, device 800 is presented in the form of a general-purpose device. Components of device 800 may include, but are not limited to: at least one processor 801, at least one memory 802, and a bus 803 connecting different system components (including memory 802 and processor 801). The memory stores program code, which, when executed by the processor, causes the processor to perform the following steps:

[0191] Collect the lane line point set of the target lane line, and use the quadratic curve equation to fit the lane line based on the collected lane line point set to obtain the curve function expression of the target lane line.

[0192] The system acquires multiple frames of images of vehicles traveling on the target lane in real time, identifies the vehicle's position in each frame, and obtains the position of the target vehicle in each frame.

[0193] By using the parameters in the curve function expression and the driving position of the target vehicle in each frame image, the positional relationship between the vehicle position in each frame image and the fitted target lane line is determined.

[0194] Based on the positional relationship between multiple frames of images, it is determined that the target vehicle has committed an illegal lane change when it is located in different lanes in different frames of images.

[0195] Bus 803 represents one or more of several bus structures, including a memory bus or memory controller, peripheral bus, processor, or a local bus using any of the various bus structures.

[0196] The memory 802 may include a readable medium in the form of volatile memory, such as random access memory (RAM) 8021 and / or cache memory 8022, and may further include read-only memory (ROM) 8023.

[0197] The memory 802 may also include a program / utility 8025 having a set (at least one) of program modules 8024, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0198] Device 800 can also communicate with one or more external devices 804 (e.g., keyboard, pointing device, etc.), and with one or more devices that enable a user to interact with device 800, and / or with any device that enables device 800 to communicate with one or more other devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 805. Furthermore, device 800 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 806. As shown, network adapter 806 communicates with other modules used with device 800 via bus 803. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with device 800, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0199] Optionally, the processor described above is used to determine the positional relationship between the vehicle position corresponding to each frame image and the fitted target lane line by utilizing the parameters in the curve function expression and the driving position of the target vehicle in each frame image, including:

[0200] For each frame of image, the positional relationship between the vehicle position in the image and the fitted target lane line is determined based on the value of det(L)*F(x,y) and the magnitude of zero.

[0201] in, F(x,y)=0 is the curve function expression of the target lane line, A′, B′, C′, D′, E′ are the parameters in the curve function expression, and (x,y) are the coordinates of the points on the target vehicle in each frame of the image.

[0202] Optionally, the processor is used to determine the positional relationship between the vehicle position corresponding to each frame image and the fitted target lane line, including:

[0203] Extract the bottom envelope of the target vehicle in each frame image and determine the center point of the bottom envelope;

[0204] By using the parameters in the curve function expression, the positional relationship between the center point of the bottom envelope and the fitted target lane line is determined.

[0205] Optionally, the processor described above is used to acquire a set of lane line points for the target lane line, including:

[0206] The target lane line image is acquired and input into the semantic segmentation model. The semantic segmentation model is then used to identify the type of the target lane line and the location information of each point in the lane line point set.

[0207] When the target lane line type is determined to be a solid line, the position information of each point in the lane line point set of the target lane line output by the semantic segmentation model is obtained.

[0208] Optionally, when there are multiple target lane lines, after the processor determines the positional relationship between the vehicle position corresponding to each frame image and the fitted target lane line, it further includes:

[0209] Based on the arrangement order of multiple target lane lines and the positional relationship between the vehicle position in each frame and the fitted target lane line, the lane in which the target vehicle is located in each frame is determined.

[0210] Optionally, after determining the positional relationship between the vehicle position corresponding to each frame image and the fitted target lane line using the parameters in the curve function expression and the driving position of the target vehicle in each frame image, the processor further includes:

[0211] Based on the positional relationship of each frame image, it is determined that the target vehicle has crossed the lane when it occupies a different lane in any frame image.

[0212] Optionally, the processor is used to determine the positional relationship between the vehicle position corresponding to each frame image and the fitted target lane line, including:

[0213] Extract the envelope regions of different headlights / tires of the target vehicle in each frame image, and determine the center point of each envelope region;

[0214] By using the parameters in the curve function expression, the positional relationship between the center point of each envelope region in each frame of the image and the fitted target lane line is determined;

[0215] Determining whether the target vehicle is located in a different lane in any frame of the image includes:

[0216] When the center point of different envelope regions of the target vehicle is located on different sides of the fitted target lane line, it is determined that the target vehicle occupies different lanes.

[0217] In some possible implementations, various aspects of the illegal lane change detection method provided in this application can also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to cause the computer device to perform the steps in the illegal lane change detection method according to the various exemplary embodiments of this application described above.

[0218] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0219] The monitoring program product of the embodiments of this application can be a portable compact disc read-only memory (CD-ROM) and include program code, and can run on a device. However, the program product of this application is not limited to this. In this document, the readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0220] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0221] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0222] Program code for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user device, partially on the user device, as a standalone software package, partially on the user device and partially on a remote device, or entirely on a remote device or server. In cases involving remote devices, the remote device can be connected to the user device via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external device (e.g., via the Internet using an Internet service provider).

[0223] It should be noted that although several units or sub-units of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0224] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0225] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0226] This application is described with reference to flowchart illustrations and block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block and / or block in the flowchart illustrations and block diagrams, as well as combinations of blocks and processes in the flowchart illustrations and block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0227] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and boxes Figure 1 The function specified in one or more boxes.

[0228] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and boxes Figure 1 The steps of the function specified in one or more boxes.

[0229] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0230] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for detecting illegal lane changing behavior, characterized in that, include: Images of the target lane lines are periodically acquired at preset intervals and input into a semantic segmentation model. The semantic segmentation model is then used to identify the type of the target lane lines and the location information of each point in the lane line point set. When the type of the target lane line is determined to be a solid line, the position information of each point in the lane line point set of the target lane line output by the semantic segmentation model is obtained, and the position information of each point forms the lane line point set. The points in the lane line point set are sampled, and the center point of the lane line point set in the same width direction is collected along the target lane line width direction. The points are then sampled uniformly along the target lane line length direction at a preset interval. Based on the sampled lane line point set, lane lines are fitted using a quadratic curve equation to obtain the curve function expression of the target lane line, and the position discriminant corresponding to the target lane line is determined based on the curve function expression. The system acquires multiple frames of images of vehicles traveling on the target lane in real time. The vehicle detection model obtained through deep learning identifies the vehicle's driving position in each frame of the image, and obtains the driving position of the target vehicle, the position information of the target vehicle's headlights, and the position information of the target vehicle's headlights and tires in each frame of the image. In addition, the license plate recognition model is used to identify the license plate of the target vehicle in each frame of the image to obtain the license plate number of the target vehicle. For any frame of the images: select a target lane line in sequence from the innermost to the outermost, or from the outermost to the innermost, and perform the following steps until the lane line where the vehicle is located is successfully determined: extract the bottom envelope of the target vehicle in each frame of the image and determine the center point of the bottom envelope; when the result of substituting the center point of the bottom envelope into the discriminant corresponding to the current target lane line is opposite to the result of substituting the center point of the bottom envelope into the discriminant corresponding to the previous target lane line, then the vehicle is determined to be located in the lane between the current target lane line and the previous target lane line; and extract the envelope regions of different headlights / tire of the target vehicle in each frame of the image and determine the center point of each envelope region respectively. For any of the envelope regions, the relative position of the center point of the envelope region to the current target lane line is determined based on the sign of the result of substituting the center point of the envelope region into the discriminant corresponding to the current target lane line. Based on the positional relationship of multiple frames of images, when it is determined that the target vehicle is located in different lanes in different frames of images, it is determined that the target vehicle has committed an illegal lane change; and, for any frame of image, when it is determined that the center point of different envelope regions of the target vehicle is located on different sides of the fitted target lane line, when it is determined that the target vehicle occupies different lanes, it is determined that the target vehicle has committed a lane crossing. The position discrimination formula corresponding to the target lane line is: ; in, , , The curve function expression for the target lane line. These are the parameters in the curve function expression. These are the coordinates of points on the target vehicle in each frame of the image.

2. A device for detecting illegal lane changing, characterized in that, The device is used to identify the license plate of the target vehicle using a license plate recognition model in each frame of the image, and obtain the license plate number of the target vehicle. And, the device includes: The fitting module is used to periodically acquire images of the target lane line at preset intervals and input them into a semantic segmentation model. The semantic segmentation model is used to identify the type of the target lane line and the position information of each point in the lane line point set. When the type of the target lane line is determined to be a solid line, the position information of each point in the lane line point set output by the semantic segmentation model is acquired, and the position information of each point forms the lane line point set. Points in the lane line point set are sampled, and the center points of the lane line point set in the same width direction are collected along the width direction of the target lane line. Uniform sampling is performed along the length direction of the target lane line at preset intervals. Based on the sampled lane line point set, a quadratic curve equation is used to fit the lane line, obtaining a curve function expression for the target lane line. The position discriminant corresponding to the target lane line is determined based on the curve function expression. The acquisition module is used to acquire multiple frames of images of vehicles traveling on the target lane in real time. A vehicle detection model obtained through deep learning is used to identify the vehicle's position in each frame, obtaining the target vehicle's position, headlight position information, and headlight and tire position information in each frame. The positional relationship determination module is used to, for any frame of the images, sequentially select a target lane line in either the innermost or outermost order, and perform the following steps until the lane line where the vehicle's position is located is successfully determined: extract the bottom envelope of the target vehicle in each frame and determine the center point of the bottom envelope; when the center point of the bottom envelope is substituted into the discriminant expression corresponding to the current target lane line... As a result, if the center point of the bottom envelope is substituted into the discriminant corresponding to the previous target lane line and the results are opposite in sign, then the vehicle position is determined to be in the lane between the current target lane line and the previous target lane line; and, the envelope regions of different headlights / tires of the target vehicle in each frame image are extracted, and the center point of each envelope region is determined respectively; for any envelope region, the relative position of the center point of the envelope region to the current target lane line is determined according to the sign of the result of the discriminant corresponding to the current target lane line when the center point of the envelope region is substituted into the discriminant corresponding to the current target lane line; for any frame image, when it is determined that the center point of different envelope regions of the target vehicle is located on different sides of the fitted target lane line, and the target vehicle occupies different lanes, it is determined that the target vehicle has committed lane-crossing behavior; The illegal lane change determination module is used to determine that the target vehicle has committed an illegal lane change when it is located in different lanes in different frames of images based on the positional relationship between the corresponding multiple frames of images. The position discrimination formula corresponding to the target lane line is: ; in, , , The curve function expression for the target lane line. These are the parameters in the curve function expression. These are the coordinates of points on the target vehicle in each frame of the image.

3. A device for detecting illegal lane changing behavior, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method of claim 1.

4. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the steps of the method of claim 1.

Images