Illegal lane change detection method, detection device and storage medium

Abstract

The present application provides a detection method, device and storage medium for illegal lane changes. The method includes: obtaining a color video stream in real time through a vehicle-mounted camera; obtaining the lane line information of the current lane, and the ID information and position information of at least one target object according to the color video stream; detecting the turn signal of the target object in real time, and judging whether the target object enters or deviates from the current lane according to the position information of the target object and the lane line information of the current lane; issuing an alarm when the target object enters or deviates from the current lane and the turn signal of the target object is not detected; when the target object enters or deviates from the current lane and the turn signal of the target object is detected, judging whether the lane line on the side where the target object enters or deviates is a solid line, and issuing an alarm when the lane line on the side where the target object enters or deviates is a solid line. The present application can complete the detection of illegal lane changes in real time and improve the detection accuracy.

Description

Technical Field

[0001] The present application relates to the field of automated driving technology, and in particular to a method, a detection device and a storage medium for detecting an illegal lane change. Background Art

[0002] With economic development and social progress, the number of cars has continued to increase. Although the increase in vehicles has improved travel convenience and passengers' travel experience in some aspects, it has also brought more and more traffic accidents, which has brought great safety hazards to people's lives and property.

[0003] Most traffic accidents are caused by bad driving habits of drivers, such as changing lanes directly without turning on the turn signal according to traffic regulations, or changing lanes on a solid line. These illegal lane changes can easily cause traffic accidents, resulting in personal or property losses. However, on roads without security monitoring systems installed, these illegal lane changes are extremely difficult to capture and record. In other words, in most cases, even if illegal lane changes are made without turning on the turn signal or changing lanes on a solid line, they cannot be promptly and effectively identified as illegal behaviors. Therefore, how to improve the detection efficiency and accuracy of illegal lane changes has become one of the problems to be solved in this field. Summary of the invention

[0004] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide a detection method, a detection device and a storage medium for illegal lane change, so as to solve the problem that the prior art cannot detect and warn the illegal lane change behavior in real time.

[0005] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present application provides a method for detecting illegal lane changes, the method comprising: acquiring a color video stream in real time through a vehicle-mounted camera; acquiring lane line information of the current lane, and ID information and position information of at least one target object according to the color video stream; detecting the turn signal of the target object in real time, and judging whether the target object enters or deviates from the current lane according to the position information of the target object and the lane line information of the current lane; issuing an alarm when the target object enters or deviates from the current lane and the turn signal of the target object is not detected; when the target object enters or deviates from the current lane and the turn signal of the target object is detected, judging whether the lane line on the side where the target object enters or deviates is a solid line, and issuing an alarm when the lane line on the side where the target object enters or deviates is a solid line.

[0006] In certain embodiments of the first aspect of the present application, the method of obtaining the lane line information of the current lane according to the color video stream includes: selecting a region of interest from the color video stream, and converting the color image information located in the region of interest into grayscale image information; generating a pixel difference image according to the grayscale image information, and scanning and comparing the pixel difference image according to a preset window, and when the grayscale difference in the preset window is greater than a predetermined threshold, setting the pixel value in the preset window to 255; dividing the region of interest into a preset number of sub-regions, and performing polar coordinate conversion on pixels with a pixel value of 255 in each sub-region; performing horizontal coordinate projection at preset angle intervals in the polar coordinate system, counting the maximum cumulative peak value of the projection, and calculating the angle and distance corresponding to the maximum peak value; converting the polar coordinate system into an image coordinate system, and performing binomial curve fitting to fit the lane line in the image; tracking the lane line according to the color video stream to obtain the lane line information of the current lane.

[0007] In certain embodiments of the first aspect of the present application, obtaining ID information and position information of at least one target object based on the color video stream includes: obtaining at least one object from the color video stream through a deep neural network target detector; obtaining the ID information and position information of the object through a multi-target tracking algorithm; and determining, based on the position information of the object and the acquired lane line information, the objects located on both sides of the current lane and closest to the lane line and the object directly in front of the current lane as the target objects.

[0008] In certain embodiments of the first aspect of the present application, the determining whether the lane line on the side where the target object enters or deviates is a solid line includes: acquiring the direction of movement of the target object based on the position information of the target object; acquiring lane line information on the side where the target object enters or deviates based on the direction of movement of the target object and the lane line information of the current lane; acquiring a lane line binary image based on the lane line information on the side where the target object enters or deviates, and setting a valid area; vertically projecting the lane line binary image within the valid area; performing histogram statistics on the projected lane line binary image; when the statistical result is greater than or equal to a ratio threshold, determining that the lane line on the side where the target object enters or deviates is a solid line, and when the statistical result is less than the ratio threshold, determining that the lane line on the side where the target object enters or deviates is a dotted line.

[0009] In certain embodiments of the first aspect of the present application, after obtaining the lane line information of the target object's entry or departure side according to the target object's movement direction and the lane line information of the current lane, it also includes: judging whether the lane line information of the target object's entry or departure side is greater than a preset lane line width; when it is greater than the preset lane line width, dividing the lane line into a first lane line and a second lane line from the center position of the lane line; obtaining a lane line binary image according to the lane line information of the target object's entry or departure side includes: binarizing the first lane line or the second lane line corresponding to the movement direction of the target object to obtain a corresponding lane line binary image.

[0010] In certain embodiments of the first aspect of the present application, the step of issuing an alarm includes: obtaining a color video stream within a preset time interval before and after the alarm occurs; and outputting the obtained color video stream within the preset time interval according to a preset transmission method.

[0011] In a second aspect of the present application, a detection device for illegal lane change is provided, characterized in that it includes: a video stream acquisition module, which is used to acquire a color video stream in real time through a vehicle-mounted camera; a lane line acquisition module, which is used to acquire lane line information of the current lane according to the color video stream; a target object acquisition module, which is used to acquire ID information and position information of at least one target object according to the color video stream; a turn signal acquisition module, which is used to detect the turn signal signal of the target object in real time; a first judgment module, which is used to judge whether the target object enters or deviates from the current lane according to the position information of the target object and the lane line information of the current lane; a second judgment module, which is used to judge whether the current lane is a solid line when the target object enters or deviates from the current lane and the turn signal signal of the target object is detected; an alarm module, which is used to issue an alarm when the target object enters or deviates from the current lane and the turn signal signal of the target object is not detected; or, when the target object enters or deviates from the current lane and the turn signal signal of the target object is detected, an alarm is issued when the current lane is a solid line.

[0012] In certain embodiments of the second aspect of the present application, the lane line acquisition module includes: an image conversion unit, which is used to select an area of ​​interest from the color video stream and convert the color image information located in the area of ​​interest into grayscale image information; a scanning and comparison unit, which is used to generate a pixel difference image based on the grayscale image information, and scan and compare the pixel difference image according to a preset window, and when the grayscale difference in the preset window is greater than a predetermined threshold, the pixel value in the preset window is set to 255; a coordinate conversion unit, which is used to divide the area of ​​interest into a preset number of sub-areas, and in each sub-area, respectively perform polar coordinate conversion on pixels with a pixel value of 255; a statistical unit, which is used to perform horizontal coordinate projection at preset angle intervals in a polar coordinate system, count the maximum peak value of the cumulative projection, and calculate the angle and distance corresponding to the maximum peak value; a fitting unit, which is used to convert the polar coordinate system into an image coordinate system, and perform binomial curve fitting to fit the lane line in the image; a tracking unit, which is used to track the lane line according to the color video stream to obtain the lane line information of the current lane.

[0013] In certain embodiments of the second aspect of the present application, the second judgment module includes: a motion direction determination unit, which is used to obtain the motion direction of the target object according to the position information of the target object; a lane line information acquisition unit, which is used to obtain the lane line information of the target object entering or deviating from the side according to the motion direction of the target object and the lane line information of the current lane; a binarization processing unit, which is used to obtain a lane line binary image according to the lane line information of the target object entering or deviating from the side, and set a valid area; a vertical projection unit, which is used to vertically project the lane line binary image in the valid area; a histogram statistics unit, which is used to perform histogram statistics on the projected lane line binary image; a lane line determination unit, which is used to determine that the current lane is a solid line when the statistical result is greater than or equal to a ratio threshold, and to determine that the current lane is a dotted line when the statistical result is less than the ratio threshold.

[0014] According to a third aspect of the present application, a storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the illegal lane change detection method as described above are implemented.

[0015] As described above, the illegal lane change detection method, detection device and storage medium of the present application have the following beneficial effects:

[0016] In this application, the color video stream obtained in real time by the on-board camera can accurately obtain the lane line information and the target object, and then combined with the real-time detection of the turn signal of the target object, when the target object enters the current lane or deviates from the current lane, if the turn signal is not detected, an alarm will be issued; or if the target object enters or deviates from the current lane and there is a turn signal, but if the current lane line is a solid line, an alarm will also be issued. In this way, even on roads where security monitoring equipment is not installed, real-time detection of illegal lane changes without turning on the turn signal or lane changes on a solid line can be achieved only through on-board cameras (such as driving recorders), thereby effectively improving the detection efficiency of illegal lane changes.

[0017] Furthermore, in the present application, the color video stream collected by the vehicle-mounted camera can obtain the lane line information of the current lane. By acquiring the lane line in real time, the efficiency and accuracy of detecting illegal lane changes are further effectively improved.

[0018] In addition, in the detection of whether the lane line is a solid line, the present application only processes the lane line on the side where the target object enters or deviates to confirm whether the target object changes lanes with a solid line, which greatly reduces the amount of processing operations and effectively improves the detection efficiency on the basis of improving the detection accuracy. At the same time, the present application adopts different processing methods when detecting lane lines of different specifications. For example, when the width of the lane line exceeds the preset width, the lane line is divided, and then combined with the movement direction of the target object, it is judged whether the corresponding lane line is a solid line, thereby realizing the detection of solid lane changes more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Shown is a flow chart of Embodiment 1 of the method for detecting illegal lane change of the present application;

[0020] Figure 2 Shown is a flow chart of Embodiment 1 of the present application for obtaining lane line information of the current lane;

[0021] Figure 3 Shown is a flow chart of Embodiment 1 of the present application for determining whether a lane line is a solid line;

[0022] Figure 4 Shown is a flow chart of Embodiment 2 of the present application for determining whether a lane line is a solid line;

[0023] Figure 5 Shown is a schematic diagram of the structure of the first embodiment of the illegal lane change detection device of the present application;

[0024] Figure 6 Shown is a structural schematic diagram of the first embodiment of the lane line acquisition module of the present application;

[0025] Figure 7 Shown is a structural diagram of Example 1 of the second judgment module of the present application. DETAILED DESCRIPTION

[0026] The following is an explanation of the implementation of the present application by means of specific embodiments. People familiar with the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification.

[0027] In the following description, with reference to the accompanying drawings, several embodiments of the present application are described. It should be understood that other embodiments may also be used, and mechanical composition, structure, electrical and operational changes may be made without departing from the spirit and scope of the present disclosure. The following detailed description should not be considered restrictive, and the scope of the embodiments of the present application is limited only by the claims of the published patents. The terms used here are only to describe specific embodiments and are not intended to limit the present application. Spatially related terms, such as "upper", "lower", "left", "right", "below", "below", "lower", "above", "upper", etc., may be used in the text to facilitate the description of the relationship between an element or feature shown in the figure and another element or feature.

[0028] Although the terms first, second, etc. are used to describe various elements in this article in some instances, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. Moreover, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising", "including" indicate that there are described features, steps, operations, elements, components, projects, kinds, and / or groups, but do not exclude the existence, occurrence or addition of one or more other features, steps, operations, elements, components, projects, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Therefore, "A, B or C" or "A, B and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C". Only when the combination of elements, functions, steps or operations is inherently mutually exclusive in some way, the exception to this definition will occur.

[0029] As described in the background technology, the prior art cannot effectively detect illegal lane-changing behaviors of vehicles on the road in real time, resulting in a large number of such illegal lane-changing behaviors, which in turn endangers people's personal and property safety. In order to detect and warn about illegal lane-changing behaviors in a timely and effective manner, the present application provides a detection method, a detection device and a storage medium for illegal lane-changing, which obtains the corresponding video stream in real time through a vehicle-mounted camera, obtains the position information, lane line information and turn signal signals of the target object in a timely manner according to the video stream, and judges in real time whether the target object has illegal lane-changing behaviors by monitoring the lane line information and turn signal signals, thereby realizing accurate detection and warning of illegal lane-changing behaviors.

[0030] See also Figure 1 , Figure 1 The flowchart of the first embodiment of the method for detecting illegal lane change of the present application is shown as follows: Figure 1 As shown, the illegal lane change detection method includes:

[0031] Step S10, obtaining a color video stream in real time through a vehicle-mounted camera;

[0032] Specifically, the vehicle-mounted camera may be a driving recorder. When the vehicle is started, the driving recorder is used to obtain a color video stream in front of the vehicle in real time. The driving recorder may include a memory card to save the obtained color video stream. Of course, the video stream during the driving process of the vehicle may also be obtained through other external or built-in vehicle-mounted cameras and other devices, and this application does not limit this.

[0033] Step S20, acquiring lane line information of the current lane, and ID information and position information of at least one target object according to the color video stream;

[0034] Unlike existing security monitoring equipment that monitors fixed areas or fixed road intersections, the lane line information in this embodiment is obtained by processing the dynamic video stream obtained by the vehicle-mounted camera. At the same time, the ID information and location information of other target objects can also be obtained in real time based on the dynamic video stream obtained by the vehicle-mounted camera. The ID information of the target object may include the license plate number of the target object, etc.

[0035] In this embodiment, obtaining ID information and position information of at least one target object according to the color video stream includes: obtaining at least one object from the color video stream through a deep neural network target detector; obtaining the ID information and position information of the object through a multi-target tracking algorithm; and determining, according to the position information of the object and the acquired lane line information, the objects located on both sides of the current lane and closest to the lane line and the object directly in front of the current lane as the target objects.

[0036] Specifically, first, the deep neural network target detector can be used to obtain the location information of at least one target (such as a vehicle, a person, etc.) from the color video stream; then the vehicle, pedestrian, and other targets obtained by the target detector are sent to the multi-target tracking algorithm to obtain the ID and location information of the tracked target; then, the lane line information is used to filter out the target vehicle closest to the left lane line, the target vehicle closest to the right lane line, and the optimal target vehicle directly in front of the current lane line according to the target, and record their corresponding location information and ID information. In other words, the target vehicle closest to the left lane line, the target vehicle closest to the right lane line, and the optimal target vehicle directly in front of the person in the current lane line are all tracked as target objects.

[0037] Please refer to Figure 2 , Figure 2 The flowchart of the first embodiment of the present application for obtaining lane line information of the current lane is shown. Figure 2 As shown, the step S20 of acquiring the lane line information of the current lane according to the color video stream includes:

[0038] Step S201 : selecting a region of interest from the color video stream, and converting the color image information in the region of interest into grayscale image information.

[0039] In this embodiment, a region of interest (ROI) can be selected from the color video stream acquired by the vehicle-mounted camera according to actual application requirements. Specifically, the area to be processed (region of interest) can be selected in a variety of ways such as a box, a circle, an ellipse, an irregular polygon, etc. By selecting the region of interest, the image or video information outside the region of interest can be effectively excluded, thereby effectively reducing the difficulty of image processing and reducing the time of image processing, thereby improving the efficiency and accuracy of subsequent image processing. In a specific application, the video stream acquired by the vehicle-mounted camera is composed of multiple consecutive frames of color images (RGB color images). In this embodiment, the RGB color image needs to be converted into a grayscale image (GRAY grayscale image) first. The specific conversion method can be implemented using existing technology, and this application does not limit this.

[0040] Step S202, generating a pixel difference image according to the grayscale image information, and scanning and comparing the pixel difference image according to a preset window, and when the grayscale difference in the preset window is greater than a predetermined threshold, setting the pixel value in the preset window to 255.

[0041] In specific applications, since the lane line and the road surface have grayscale value differences, the grayscale image information can be processed by horizontal line-by-line scanning to generate a pixel difference image. In addition, the preset window can be set according to the national lane line standard, for example, the preset window is set to the lane line width, and then the pixel difference image is scanned line by line for comparison. If the grayscale difference in a preset window exceeds a predetermined threshold, the pixel value in the preset window is set to 255 (white), and the pixel value outside the preset window is set to 0 (black). The predetermined threshold can be adjusted or optimized according to the specific application scenario, and it should not limit the scope of protection of this application.

[0042] Step S203: Divide the region of interest into a preset number of sub-regions, and perform polar coordinate conversion on pixels with a pixel value of 255 in each sub-region.

[0043] In this embodiment, the preset number can be 6, that is, the region of interest can be divided into 6 regions, and polar coordinate conversion is performed on pixels with a pixel value of 255 (white lane line region) in each region. The preset number can be adjusted according to actual conditions, and this application does not impose any restrictions on this.

[0044] Step S204, performing horizontal coordinate projection at preset angle intervals in the polar coordinate system, counting the maximum cumulative peak value of the projection, and calculating the angle and distance corresponding to the maximum peak value.

[0045] In this embodiment, the preset angle may be 1°, 2° or other degrees.

[0046] Step S205 , converting the polar coordinate system into the image coordinate system, and performing binomial curve fitting to fit the lane lines in the image.

[0047] Step S206: Track the lane line according to the color video stream to obtain lane line information of the current lane.

[0048] The lane line information of the current lane includes two lane line information, namely the left lane line information of the current lane and the right lane line information of the current lane. In this application, the dynamic color video stream obtained in real time by the vehicle-mounted camera can realize real-time detection of lane lines, which can not only improve the detection efficiency of illegal lane changes, but also can provide real-time reminders to drivers of lane deviation, thereby improving their safety.

[0049] Continue to refer Figure 1 After obtaining the lane line information of the current lane and the relevant information of the target object through step S20, step S30 is executed to detect the turn signal of the target object in real time.

[0050] In this embodiment, the turn signal of the target object is detected by the target detector. When the target detector detects the turn signal, it is determined that the turn signal of the target object is turned on. Otherwise, it is determined that the turn signal of the target object is not turned on.

[0051] When the turn signal of the target object is detected, the turn signal enable signal corresponding to the target object is set to ON; otherwise, it is set to OFF. For example, when the turn signal is detected on the target vehicle closest to the left lane line, the turn signal enable signal of the left vehicle is set to ON, otherwise it is set to OFF; similarly, when the turn signal is detected on the target vehicle closest to the right lane line, the turn signal enable signal of the right vehicle is set to ON, otherwise it is set to OFF.

[0052] Step S40, determining whether the target object enters or deviates from the current lane; specifically, determining whether the target object enters or deviates from the current lane based on the position information of the target object and the lane line information of the current lane.

[0053] In this embodiment, by continuously tracking the position information of the target object, the moving direction of the target object can be monitored in real time, and the moving direction of the target object can be effectively determined whether the target object enters the current lane or leaves the current lane. For example, when it is detected that the target object in the left lane enters the current lane from the adjacent left lane, it is determined that the target object enters the current lane; when it is detected that the target object in front of the current lane suddenly turns and leaves the current lane, it is determined that the target object deviates from the current lane.

[0054] Continue to refer Figure 1 , when the target object enters or deviates from the current lane and the turn signal of the target object is not detected, executing step S50 to issue an alarm;

[0055] When the target object enters or deviates from the current lane and the turn signal of the target object is detected, step S60 is executed to determine whether the lane line on the side where the target object enters or deviates is a solid line;

[0056] When the lane line on the side where the target object enters or deviates is a solid line, step S50 is executed to issue an alarm.

[0057] In this embodiment, the step of issuing an alarm in step S50 includes: acquiring a color video stream within a preset time interval before and after the alarm occurs; and outputting the acquired color video stream within the preset time interval according to a preset transmission mode.

[0058] Specifically, a certain time interval can be preset according to actual needs. For example, the preset time interval can be set to 6 seconds; the transmission method can also be preset, such as transmitting to a mobile terminal such as a mobile phone via Bluetooth. For example, when an alarm occurs, the video stream of the first 3 seconds when the alarm occurs and the last 3 seconds when the alarm occurs can be edited, and the 6-second video stream can be transmitted to the user-specified mobile phone via Bluetooth or other wireless transmission methods. It should be noted that the preset time interval can also be set to other settings (such as 5 seconds, 8 seconds, 10 seconds, etc.), and the time intervals before and after the alarm can be the same or different. For example, the video stream of 4 seconds before the alarm occurs and 2 seconds after the alarm occurs (a total of 6 seconds) can be transmitted. The preset time interval and the preset transmission method are not limited to the above examples. In addition, the alarm method of step S50 can also be a variety of methods such as voice prompts and highlight prompts, which are not limited by this application.

[0059] In this embodiment, the color video stream in front of the current vehicle is obtained in real time through the vehicle-mounted camera, and the turn signal of the target object is detected in real time. The lane line information of the current vehicle can be obtained in real time based on the color video stream, and a real-time alarm is issued when the target object enters or deviates from the current lane and its turn signal is not detected, or when the target object enters or deviates from the current lane, although its turn signal is detected, it will also issue an alarm when changing lanes on a solid line. In this way, the detection and alarm of lane changes without turning on the turn signal or on a solid line can be realized during the driving process of the vehicle, thereby effectively improving the detection efficiency of illegal lane changes.

[0060] refer to Figure 3 , Figure 3 The flowchart of the first embodiment of the present application for determining whether the lane line is a solid line is shown. Figure 3 As shown, the determination of whether the lane line on the side where the target object enters or deviates is a solid line includes:

[0061] Step S601, acquiring the moving direction of the target object according to the position information of the target object;

[0062] Step S602, obtaining lane line information of the target object's entry or departure side according to the target object's moving direction and the lane line information of the current lane;

[0063] Step S603, obtaining a lane line binary image according to the lane line information of the target object entering or departing from the side, and setting a valid area;

[0064] In practical applications, the lane line information of the current lane includes two lane line information, namely, the left lane line information of the current lane and the right lane line information of the current lane. Assuming that the target object A is located in the left lane adjacent to the current lane, when the target object A turns right into the current lane, the left lane line information of the current lane is used as the lane line information of the entry side, and step S603 is executed, that is, the left lane line information of the current lane is binarized to obtain the lane line binarization image.

[0065] Assuming that the target object B is located in the right lane adjacent to the current lane, when the target object B turns left into the current lane, the right lane line information of the current lane is used as the lane line information of the entering side, and step S603 is executed, that is, the right lane line information of the current lane is binarized to obtain the lane line binary image.

[0066] Assuming that the target object C is located directly in front of the current lane, when the target object C turns right and deviates from the current lane, the right lane line information of the current lane is used as the lane line information of the deviating side, and step S603 is executed, that is, the right lane line information of the current lane is binarized to obtain the lane line binary image.

[0067] Step S604, vertically projecting the lane line binary image in the effective area;

[0068] Step S605, performing histogram statistics on the projected lane line binary image;

[0069] Step S606: When the statistical result is greater than or equal to the ratio threshold, the lane line on the side where the target object enters or deviates is determined to be a solid line; when the statistical result is less than the ratio threshold, the lane line on the side where the target object enters or deviates is determined to be a dotted line.

[0070] In this embodiment, the ratio threshold may be set to 0.5 according to the national lane line standard. Of course, the ratio threshold may also be adjusted according to other application scenarios.

[0071] In this embodiment, by processing and judging whether the lane line on the side where the target object enters or deviates is a solid lane change line, not only the processing calculation is reduced, but also the detection efficiency is greatly improved.

[0072] refer to Figure 4 , Figure 4 The flowchart of the second embodiment of the present application for determining whether the lane line is a solid line is shown. Figure 3The difference of the first embodiment shown is that the process of determining whether the lane line on the side where the target object enters or deviates is a solid line, after obtaining the lane line information on the side where the target object enters or deviates according to the moving direction of the target object and the lane line information of the current lane in step S602, further includes:

[0073] Step S607, determining whether the lane line information of the target object entering or departing from the side is greater than a preset lane line width; if greater than the preset lane line width, executing step S608, dividing the lane line into a first lane line and a second lane line from the center position of the lane line;

[0074] The process of obtaining a lane line binary image according to the lane line information of the target object entering or departing from the side of the target object in step S603 specifically includes: binarizing the first lane line or the second lane line corresponding to the moving direction of the target object to obtain a corresponding lane line binary image, and setting a valid area; then executing steps S604 to S606. Figure 3 The first embodiment shown is similar and will not be described again here.

[0075] On actual roads, there are many types of lane lines. In order to more accurately detect solid line lane change behavior, the lane line width can be judged first. When the lane line width exceeds the preset lane line width, the lane line is divided into two lane lines, so that it can be more accurately judged whether it is a solid line lane change behavior, further improving the accuracy of solid line lane change detection.

[0076] For example, assuming that the target object A is located in the left lane adjacent to the current lane, when the target object A turns right to enter the current lane, it can be known that the target object A is moving to the right, and the left lane line information of the current lane is used as the lane line information on the entering side. Assuming that the left lane line width of the current lane is greater than the preset lane line width according to step S607, step S608 is executed to divide the left lane line of the current lane from the center position into a first lane line and a second lane line. For example, the lane line on the left side of the center position can be defined as the first lane line, and the lane line on the right side of the center position can be defined as the second lane line (of course, the first lane line and the second lane line can also be defined in the opposite way, and this application does not limit this). Since the target object A is moving to the right, it is necessary to determine whether the first lane line is a solid line, that is, steps S603 to S606 are subsequently executed on the first lane line.

[0077] Assume that the target object D is located in front of the current lane. When the target object D turns left and deviates from the current lane, it can be known that the target object D is moving to the left, and the left lane line information of the current lane is used as the lane line information of the deviating side. Assuming that the left lane line width of the current lane is greater than the preset lane line width according to step S607, step S608 is executed to divide the left lane line of the current lane from the center position into a first lane line and a second lane line. For example, the first lane line can be defined as the left side of the center position, and the second lane line can be defined as the right side of the center position (of course, the first lane line and the second lane line can also be defined in the opposite way, and this application does not limit this). Since the target object D is moving to the left, it is necessary to determine whether the second lane line is a solid line, that is, steps S603 to S606 are subsequently executed on the second lane line.

[0078] Please refer to Figure 5 , Figure 5 The structure diagram of the first embodiment of the illegal lane change detection device of the present application is shown. Figure 5 As shown, the illegal lane change detection device includes:

[0079] The video stream acquisition module 10 is used to acquire the color video stream in real time through the vehicle-mounted camera;

[0080] A lane line acquisition module 20, used to acquire lane line information of the current lane according to the color video stream;

[0081] A target object acquisition module 30, configured to acquire ID information and location information of at least one target object according to the color video stream;

[0082] A turn signal acquisition module 40, used for detecting the turn signal of the target object in real time;

[0083] A first judgment module 50, configured to judge whether the target object enters or deviates from the current lane according to the position information of the target object and the lane line information of the current lane;

[0084] A second judgment module 60 is used to judge whether the current lane is a solid line when the target object enters or deviates from the current lane and a turn signal of the target object is detected;

[0085] The warning module 70 is used to issue a warning when the target object enters or deviates from the current lane and the turn signal of the target object is not detected; or when the target object enters or deviates from the current lane and the turn signal of the target object is detected, and the current lane is a solid line, to issue a warning.

[0086] Please refer to Figure 6 , Figure 6 The structure diagram of the lane line acquisition module of the present application is shown as a first embodiment. Figure 6 As shown, the lane line acquisition module includes:

[0087] An image conversion unit 201 is used to select a region of interest from the color video stream and convert the color image information in the region of interest into grayscale image information;

[0088] A scanning and comparison unit 202 is used to generate a pixel difference image according to the grayscale image information, and scan and compare the pixel difference image according to a preset window, and when the grayscale difference in the preset window is greater than a predetermined threshold, set the pixel value in the preset window to 255;

[0089] A coordinate conversion unit 203, used for dividing the region of interest into a preset number of sub-regions, and performing polar coordinate conversion on pixels with a pixel value of 255 in each sub-region;

[0090] A statistical unit 204 is used to perform horizontal coordinate projection at preset angle intervals in a polar coordinate system, to count the maximum peak value accumulated by the projection, and to find the angle and distance corresponding to the maximum peak value;

[0091] A fitting unit 205, used for converting the polar coordinate system into the image coordinate system and performing binomial curve fitting to fit the lane line in the image;

[0092] The tracking unit 206 is used to track the lane line according to the color video stream to obtain lane line information of the current lane.

[0093] Please refer to Figure 7 , Figure 7 Shown is a structural diagram of Example 1 of the second judgment module of the present application.

[0094] like Figure 7 As shown, the second judgment module includes:

[0095] A motion direction determining unit 601 is used to obtain the motion direction of the target object according to the position information of the target object;

[0096] The lane line information acquisition unit 602 is used to acquire the lane line information of the target object's entry or departure side according to the moving direction of the target object and the lane line information of the current lane;

[0097] A binarization processing unit 603 is used to obtain a lane line binarization image according to the lane line information of the target object entering or departing from the side, and set a valid area;

[0098] A vertical projection unit 604 is used to perform vertical projection on the lane line binary image in the effective area respectively;

[0099] A histogram statistics unit 605 is used to perform histogram statistics on the projected lane line binary image;

[0100] The lane line determination unit 606 is used to determine that the current lane is a solid line when the statistical result is greater than or equal to the ratio threshold, and to determine that the current lane is a dotted line when the statistical result is less than the ratio threshold.

[0101] In another embodiment, the second judgment module may further include: a lane line width judgment unit and a lane line division unit. The lane line width judgment unit is connected to the lane line information acquisition unit 602, and is used to judge whether the lane line information of the target object entering or deviating from the side is greater than the preset lane line width; the lane line division unit is used to divide the lane line into a first lane line and a second lane line from the center position of the lane line when it is greater than the preset lane line width. The binarization processing unit 603 is used to binarize the first lane line or the second lane line corresponding to the moving direction of the target object to obtain a corresponding lane line binarization image.

[0102] The processing process of the illegal lane change detection device of the present application can refer to the description of the aforementioned illegal lane change detection method, which will not be repeated here.

[0103] Accordingly, the present application also provides a storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned illegal lane change detection method are implemented. The specific implementation process can refer to the specific description of the aforementioned illegal lane change detection method, which will not be repeated here.

[0104] It should be noted that the storage medium may include a USB flash drive, a disk, a floppy disk, an optical disk, a DVD, a hard disk, a flash memory, a CF card, an SD card, an MMC card, a SM card, etc., and this application does not limit this. In addition, the storage medium may be used as a separate component or as a partial component in an electronic device. For example, the electronic device may include but is not limited to a laptop computer, a tablet computer, a mobile phone, a smart phone, a media player, a personal digital assistant (PDA), etc., and also includes a combination of two or more thereof. The electronic device may include a memory, a memory controller, one or more processing units (CPUs), a peripheral interface, an RF circuit, an audio circuit, a speaker, a microphone, an input / output (I / O) subsystem, a touch screen, other output or control devices, and an external port. These components communicate via one or more communication buses or signal lines. The electronic device also includes a power supply system for powering various components. The power supply system may include a power management system, one or more power supplies (e.g., batteries, alternating current (AC)), a charging system, a power failure detection circuit, a power converter or inverter, a power status indicator (e.g., a light emitting diode (LED)), and any other components associated with power generation, management, and distribution in a portable device.

[0105] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A method for detecting illegal lane change, characterized in that: include: Obtain color video stream in real time through the vehicle-mounted camera; Acquire lane line information of a current lane and ID information and position information of at least one target object according to the color video stream; The method of obtaining the lane line information of the current lane according to the color video stream includes: selecting an area of ​​interest from the color video stream, and converting the color image information in the area of ​​interest into grayscale image information; generating a pixel difference image according to the grayscale image information, and scanning and comparing the pixel difference image according to a preset window, and when the grayscale difference in the preset window is greater than a predetermined threshold, setting the pixel value in the preset window to 255; dividing the area of ​​interest into a preset number of sub-areas, and performing polar coordinate conversion on pixels with a pixel value of 255 in each sub-area; performing horizontal coordinate projection at preset angle intervals in the polar coordinate system, counting the maximum peak value of the cumulative projection, and calculating the angle and distance corresponding to the maximum peak value; converting the polar coordinate system into an image coordinate system, and performing binomial curve fitting to fit the lane line in the image; tracking the lane line according to the color video stream to obtain the lane line information of the current lane; Detect the turn signal of the target object in real time, and determine whether the target object enters or deviates from the current lane according to the position information of the target object and the lane line information of the current lane; When the target object enters or deviates from the current lane and the turn signal of the target object is not detected, an alarm is issued; When the target object enters or deviates from the current lane and the turn signal of the target object is detected, it is determined whether the lane line on the side where the target object enters or deviates is a solid line. When the lane line on the side where the target object enters or deviates is a solid line, an alarm is issued. The determination of whether the lane line on the side where the target object enters or deviates is a solid line includes: acquiring the moving direction of the target object according to the position information of the target object; acquiring the lane line information on the side where the target object enters or deviates according to the moving direction of the target object and the lane line information of the current lane; acquiring a lane line binary image according to the lane line information on the side where the target object enters or deviates, and setting an effective area; vertically projecting the lane line binary image within the effective area; performing histogram statistics on the projected lane line binary image; when the statistical result is greater than or equal to a ratio threshold, determining that the lane line on the side where the target object enters or deviates is a solid line, and when the statistical result is less than the ratio threshold, determining that the lane line on the side where the target object enters or deviates is a dotted line.

2. The method according to claim 1, characterized in that The step of acquiring ID information and location information of at least one target object according to the color video stream comprises: Acquire at least one object from the color video stream using a deep neural network object detector; Obtaining the ID information and location information of the object through a multi-target tracking algorithm; According to the position information of the object and the acquired lane line information, the objects located on both sides of the current lane and closest to the lane line and the object directly in front of the current lane are determined as the target objects.

3. The method according to claim 1, characterized in that: After acquiring the lane line information of the target object's entry or departure side according to the moving direction of the target object and the lane line information of the current lane, the method further includes: Determine whether the lane line information of the target object entering or departing from the side is greater than a preset lane line width; When the lane width is greater than the preset lane width, the lane is divided into a first lane and a second lane from the center of the lane; The acquiring of a lane line binary image according to lane line information of the target object entering or departing from the side includes: binarizing a first lane line or a second lane line corresponding to the moving direction of the target object to acquire a corresponding lane line binary image.

4. The method according to claim 1, characterized in that The steps of issuing an alarm include: Obtaining a color video stream within a preset time interval before and after an alarm occurs; The acquired color video stream within the preset time interval is output according to a preset transmission mode.

5. An illegal lane change detection device, characterized in that: include: Video stream acquisition module, used to acquire color video stream in real time through the vehicle-mounted camera; A lane line acquisition module, used to acquire lane line information of the current lane according to the color video stream; The lane line acquisition module includes: an image conversion unit, which is used to select an area of ​​interest from the color video stream and convert the color image information located in the area of ​​interest into grayscale image information; a scanning and comparison unit, which is used to generate a pixel difference image according to the grayscale image information, and scan and compare the pixel difference image according to a preset window, and when the grayscale difference in the preset window is greater than a predetermined threshold, the pixel value in the preset window is set to 255; a coordinate conversion unit, which is used to divide the area of ​​interest into a preset number of sub-areas, and perform polar coordinate conversion on pixels with a pixel value of 255 in each sub-area; a statistical unit, which is used to perform horizontal coordinate projection at preset angle intervals in a polar coordinate system, count the maximum peak value accumulated by the projection, and calculate the angle and distance corresponding to the maximum peak value; a fitting unit, which is used to convert the polar coordinate system into an image coordinate system, and perform binomial curve fitting to fit the lane line in the image; a tracking unit, which is used to track the lane line according to the color video stream to obtain the lane line information of the current lane; A target object acquisition module, used to acquire ID information and location information of at least one target object according to the color video stream; A turn signal acquisition module, used for detecting the turn signal of the target object in real time; A first judgment module, used for judging whether the target object enters or deviates from the current lane according to the position information of the target object and the lane line information of the current lane; A second judgment module is used to judge whether the current lane is a solid line when the target object enters or deviates from the current lane and the turn signal of the target object is detected; the second judgment module includes: a movement direction determination unit, used to obtain the movement direction of the target object according to the position information of the target object; a lane line information acquisition unit, used to obtain the lane line information of the target object entering or deviating from the side according to the movement direction of the target object and the lane line information of the current lane; a binarization processing unit, used to obtain a lane line binary image according to the lane line information of the target object entering or deviating from the side, and set a valid area; a vertical projection unit, used to vertically project the lane line binary image in the valid area respectively; a histogram statistics unit, used to perform histogram statistics on the projected lane line binary image; a lane line determination unit, used to determine that the current lane is a solid line when the statistical result is greater than or equal to a ratio threshold, and determine that the current lane is a dotted line when the statistical result is less than the ratio threshold; The warning module is used to issue a warning when the target object enters or deviates from the current lane and the turn signal of the target object is not detected; or when the target object enters or deviates from the current lane and the turn signal of the target object is detected, the warning is issued when the current lane is a solid line.

6. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images