Vehicle detection method, lane line detection method and vehicle positioning method

By combining CSPDarknet and EfficientNet networks, feature pyramids and segmented heads in highway construction environments, accurate detection of vehicles and lane lines is achieved, solving the problems of low detection accuracy and limited computing resources in the existing technology, and improving Detection accuracy and robustness.

CN119027887APending Publication Date: 2024-11-26SHANXI JIAOJIAN MASCH HUAYANGHU CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411017955.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-29
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

In highway construction environment, existing vehicle detection and lane line detection algorithms are difficult to achieve accurate judgment, resulting in low accuracy of early warning systems, especially when computing resources and equipment power consumption are limited, and the vehicle speed is fast. The accuracy of detection of small objects at the far end of the field of vision is low.

Method used

Using CSPDarknet backbone network and EfficientNet backbone network to combine feature pyramids and prediction/segment heads, the detection of vehicles and lane lines is realized through feature cropping and bidirectional feature pyramid interaction, combining pixel-level semantic segmentation and instance-level lane line detection to improve detection accuracy Sex and robustness.

Benefits of technology

Effectively reduce the calculation amount of the vehicle detection network, realize real-time operation, improve the accuracy of long-distance small object detection and the robustness of lane line detection, overcome the problems of occlusion and poor generalization, and improve the highway scenario. detection capability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027887A_ABST
    Figure CN119027887A_ABST
Patent Text Reader

Abstract

The invention relates to a vehicle detection method, a lane line detection method and a vehicle positioning method. The invention firstly discloses a vehicle detection network and a lane line detection network. The vehicle detection network comprises a CSPDarknet backbone network, a feature pyramid and a prediction head which are connected in sequence, and the lane line detection network comprises an OfficientNet backbone network, a bidirectional feature pyramid and a segmentation head; secondly, a parking space positioning method is disclosed based on the vehicle detection network and the lane line detection network, and the overall vehicle positioning efficiency is improved through asynchronous processing of a vehicle detection task and a lane line detection task.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle positioning, and more specifically to a vehicle detection method, a lane line detection method and a vehicle positioning method. Background Art

[0002] my country has a long highway mileage, a large highway maintenance task, and a complex construction environment. The probability of accidents occurring during highway maintenance and construction is high, and the severity of accidents is high. A safety warning system is urgently needed to improve the safety of highway maintenance and construction. At present, domestic road construction warnings mainly rely on radar to measure vehicle speed and use cameras to record the scene. However, this solution cannot accurately judge the lane where the dangerous vehicle is located, resulting in a low accuracy rate of the warning system for potential dangers.

[0003] By introducing an intelligent processing algorithm based on visual information to detect vehicles and lane lines, it is possible to accurately determine whether there are vehicles in the construction lane, thereby improving the accuracy of early warning.

[0004] In academia, the current vehicle target detection algorithm is relatively mature. However, in the highway application scenario, computing resources and device power consumption are strictly limited. In addition, the vehicles in this scenario are fast. In order for the early warning algorithm to leave sufficient reaction time for construction workers, there are high requirements for the detection accuracy of small targets at the far end of the field of view. In order to meet the requirements of the highway scenario, the detection algorithm usually adapts through the following methods:

[0005] 1) Reduce image resolution: The purpose is to reduce the amount of algorithm calculation. This method mainly includes cropping images and downsampling of image pixels. Cropping images will crop the original image according to the detection area preset by the designer to reduce the input size. Although this method can reduce the amount of calculation, it will also cause information loss. Although image downsampling retains the overall image of the picture, it achieves the purpose of reducing resolution by losing the detailed information of the image. This method obviously affects the network accuracy due to the loss of detailed information.

[0006] 2) Increase the network structure: The network's ability to detect small targets at the far end of the field of view can be enhanced by adding additional auxiliary detection modules or introducing additional prior information such as anchor frame designs of more scales. Obviously, since these two solutions introduce additional network structures, while improving the small target detection performance, they will also further increase the network calculation amount.

[0007] The lane detection algorithm also faces the same problems as the vehicle detection algorithm, namely, the algorithm's operating efficiency and the difficulty in detecting the distant lane lines. The current mainstream practices are mainly as follows:

[0008] 1) Based on the segmentation paradigm: The lane detection algorithm based on the segmentation paradigm extracts the high-dimensional semantic information of the image through a neural network, and then uses the segmentation head to classify each pixel of the input image to obtain a pixel-level lane mask. This method is more accurate than the following two methods when detecting lanes because it divides the lane area based on pixel categories, but it is also difficult to solve the problem of lane area being blocked in the visual image. In addition, due to the distortion of the distant view in the picture caused by the depth of field effect, segmentation methods often cannot detect lane lines extending to the far end of the field of view.

[0009] 2) Based on the modeling paradigm: The lane detection algorithm based on the modeling paradigm first models the lane as a set of parameter representations. Then, the high-dimensional features in the image are extracted through a neural network, and each modeling parameter of the lane is obtained through a head output in a specific format, thereby obtaining the mathematical representation of the lane. However, since a large amount of subjective prior information is often introduced in the lane modeling process, the algorithm has poor generalization ability in different scenarios.

[0010] 3) Based on traditional image algorithms: Lane detection technologies based on traditional image algorithms mainly include edge detection and Hough transform. Edge detection extracts the strong gradient part of the image by convolution of the detection operator to determine the boundary edge between the lane line and the road; Hough transform parameterizes the straight line in the edge image to determine the most likely straight line in the parameter space. A major feature of traditional image algorithms is the introduction of a large amount of artificially provided prior information, which leads to the limited generalization ability of traditional algorithms. The detection effect is poor in cases where there is vehicle occlusion. Summary of the invention

[0011] In view of this, in order to at least partially solve the above technical problems, the present invention provides a vehicle detection method, a lane line detection method and a vehicle positioning method;

[0012] In order to achieve the above object, the present invention adopts the following technical solution:

[0013] First, the present application discloses a vehicle detection method, comprising:

[0014] Detecting the frame images of the monitoring video using a vehicle detection network to obtain a vehicle feature map, and screening the vehicle feature map to obtain a vehicle detection result;

[0015] The vehicle detection network includes a CSPDarknet backbone network, a feature pyramid and a prediction head connected in sequence;

[0016] CSPDarknet backbone network, used to extract features of different sizes of frame images and input the output features of multiple blocks into the feature pyramid;

[0017] Feature pyramid, used to interact the output features of multiple blocks, including a first link for propagating low-size features to high-size features output by multiple blocks and a second link for propagating high-size features to low-size features. After the same-size features in the first link and the second link are fused in series, the interacted features are output;

[0018] The prediction head includes multiple convolutional layers, each of which is used to output a vehicle feature map based on an interactive feature.

[0019] Preferably, after acquiring the frame image of the monitoring video, downsampling is first performed, and pixel strips are added on the upper and lower sides of the image to obtain an image of a desired resolution.

[0020] As a preference, the output features of the CSPDarknet backbone network are cropped first to avoid the loss of the upper 1 / 3 of the image information and reduce the amount of network computation.

[0021] Preferably, the vehicle feature map includes vehicle detection results for distant, mid and close views, and each vehicle detection result includes 7 channel data, wherein the first and second digits are vehicle center coordinate correction coefficients, the third and fourth digits are vehicle length and width correction coefficients, the fifth and sixth digits are confidence levels for whether the vehicle is a small vehicle or a large vehicle, and the seventh digit is confidence level for whether the vehicle is a positive example.

[0022] Preferably, screening the vehicle characteristic map to obtain a vehicle detection result includes:

[0023] A confidence level is set, and the output results of multiple convolutional layers are screened according to the confidence level, and a non-maximum suppression module is used to remove repeated detections of the same vehicle in the preliminary screening results.

[0024] Preferably, the convolution layer modifies the feature based on the correction coefficient according to the following formula:

[0025] b x =(2×r x -0.5+g i )×s i

[0026] b y =(2×r y -0.5+g i )×s i

[0027]

[0028] Where b x , b y , b h , b wis the center coordinate and longitudinal and lateral length of the predicted vehicle, r x 、r y 、r h 、r w are the corresponding correction systems, g i 、s i Represents the size of the preset grid corresponding to this level of feature map and the scaling factor of this level of feature map compared to the original image size, Represents the vertical and horizontal lengths of the preset anchor box.

[0029] Secondly, the present application discloses a lane line detection method, comprising:

[0030] The lane line detection network is used to detect the frame image of the monitoring video to obtain the segmentation mask, and the lane line is detected based on the segmentation mask and the lane line instance;

[0031] The lane detection network includes an EfficientNet backbone network, a bidirectional feature pyramid, and a segmentation head;

[0032] The EfficientNet backbone network is used to extract features of different sizes of frame images and input the features output from multiple different stages into the bidirectional feature pyramid;

[0033] A bidirectional feature pyramid is used to fuse features output from multiple different stages, including a first path and a second path for propagating high-size features output from multiple different stages to low-size features, and features of the same size in the first path and the second path are directly connected, and multiple fused features are sequentially output through the second path; it also includes a third path for propagating from low-size features in intermediate features to high-size features, and the intermediate features do not include the highest-size features and the lowest-size features in the output features of different stages; the third path is serially connected between corresponding-size features in the first path and the second path, and the lowest-size feature in the third path is integrated into the lowest-size feature in the first path, and the highest-size feature in the third path is propagated to the highest-size feature in the second path;

[0034] The segmentation head is used to perform convolution upsampling and weighted averaging on the fused features combined with the features initially extracted by the EfficientNet backbone network, and then output the segmentation mask.

[0035] Preferably, detecting the lane line based on the segmentation mask and the lane line instance includes:

[0036] The area corresponding to the channel with the largest confidence score in the segmentation mask is used as the area category of the current pixel position to obtain the lane line area mask;

[0037] The lane line area mask is matched with the preset lane line instance to obtain the first lane line in the lane line; the direction of the second lane line in the lane line relative to the first lane line is obtained according to the shooting position; and the second lane line is determined.

[0038] Preferably, the method for generating the preset lane line instance is:

[0039] Based on the bottom edge of the frame image, get equally spaced sampling points.

[0040] Taking the sampling point as the starting point and starting from a preset angle, a lane line instance is generated every n pixels on the vertical axis.

[0041] Again, the present application also discloses a vehicle positioning method, which at least applies any of the above vehicle detection methods, or applies the above lane line detection method to achieve vehicle positioning, and the positioning steps include:

[0042] S1, initialize the vehicle detection network and lane line detection network;

[0043] S2, obtaining the current frame image in the monitoring video, detecting the vehicle using the vehicle detection network, and obtaining a vehicle detection result;

[0044] S3: Determine whether it is the first lane line detection.

[0045] If yes, determine whether there is a close-up vehicle in the detected vehicle detection result. If yes, return to S2 to obtain the next frame image and re-detect the vehicle. If no, perform lane line detection and determine the lane where each vehicle is located based on the lane line detection result and the vehicle detection result.

[0046] If not, determine whether the interval from the last lane line detection reaches the preset frame interval. If not, directly determine the lane where each vehicle is located based on the lane line detection results and the vehicle detection results. If yes, determine whether there is a vehicle in the vehicle detection results. If so, first determine the lane where each vehicle is located based on the lane line detection results and the vehicle detection results. If not, re-detect the lane lines, and then determine the lane where each vehicle is located based on the lane line detection results and the vehicle detection results.

[0047] Preferably, the lane where each vehicle is located is determined according to the lane line detection result and the vehicle detection result, and the process includes:

[0048] The bottom center of the vehicle detection frame in the vehicle detection result is used as the vehicle position in the image.

[0049] Based on the ordinate of the center of the bottom edge, determine the point with the same ordinate on the detected lane line.

[0050] The vehicle is positioned based on the points on the lane line with the same ordinate and the center of the bottom edge.

[0051] Preferably, it is applied to safety early warning of highway maintenance and construction;

[0052] And based on the vehicle positioning results, determine whether there is a vehicle in the construction lane.

[0053] It can be seen from the above technical solutions that the present invention discloses a vehicle detection method, a lane line detection method and a vehicle positioning method, which are intended to solve the current problems of difficulty in detecting small targets, poor generalization of lane line detection and poor real-time performance of the algorithm; compared with the prior art,

[0054] First, the vehicle detection algorithm provided by the present invention effectively reduces the computational complexity of the vehicle detection network through feature clipping, so that the network can be implemented in real time on the edge computing platform; it can not only improve the processing speed and realize the real-time detection of vehicle targets, but also enhance the detection capability of small targets at long distances; improve the accuracy of small target vehicle detection at long distances, especially for highway scenarios;

[0055] Secondly, the lane line detection algorithm in this application combines pixel-level semantic segmentation with instance-level lane line detection. On the basis of maintaining the strong generalization of the segmentation algorithm, it introduces an instanced lane line model to improve the ability of far-end fuzzy lane lines. It not only overcomes the vehicle occlusion problem and realizes accurate lane line detection, but also improves the robustness and generalization ability of lane line detection in highway scenarios;

[0056] Thirdly, the vehicle positioning method processes the lane line detection task and the lane detection task asynchronously, and finally locates the vehicle in the lane based on the position information obtained from the lane line and vehicle detection, which can effectively improve the overall efficiency of the algorithm and avoid unnecessary computational overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0058] Figure 1 A schematic diagram of the vehicle detection network structure provided by the present invention;

[0059] Figure 2 A schematic diagram of a characteristic pyramid structure provided by the present invention;

[0060] Figure 3 A schematic diagram of the lane detection network structure provided by the present invention;

[0061] Figure 4 A schematic diagram of a bidirectional feature pyramid structure provided by the present invention;

[0062] Figure 5 A schematic diagram of a method for instantiating a second lane line provided by the present invention;

[0063] Figure 6 A schematic diagram for generating a preset lane line example provided by the present invention;

[0064] Figure 7 This is a flow chart of the vehicle positioning method provided by the present invention. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] Embodiment 1

[0067] The embodiment of the present invention discloses a vehicle detection method, comprising:

[0068] Detecting the frame images of the monitoring video using a vehicle detection network to obtain a vehicle feature map, and screening the vehicle feature map to obtain a vehicle detection result;

[0069] The vehicle detection network includes a CSPDarknet backbone network, a feature pyramid and a prediction head connected in sequence;

[0070] CSPDarknet backbone network, used to extract features of different sizes of frame images and input the output features of multiple blocks into the feature pyramid;

[0071] Feature pyramid, used to interact with the output features of multiple blocks;

[0072] The prediction head includes multiple convolutional layers, each of which is used to output a vehicle feature map based on an interactive feature.

[0073] In a preferred embodiment, the specific steps include:

[0074] 1. After obtaining the frame image of the monitoring video, downsample it first, and add pixel strips on the upper and lower sides of the image to obtain an image with the required resolution. In this embodiment, read the latest frame transmitted by the video sensor, downsample the RGB three-channel original image with a resolution of 3×720×1280 (the dimensional information hereafter does not include the batchsize dimension) to 3×360×640, and then add pixel strips with a width of 12 pixels and a value of (114, 114, 114) on the upper and lower sides of the downsampled image, so that the image resolution of the input model is 3×384×640.

[0075] 2. Input the preprocessed image into the vehicle detection network, use the vehicle detection network to detect, and obtain the vehicle feature map; in this embodiment, the vehicle detection network refers to Figure 1 , including CSPDarknet backbone network, feature pyramid and prediction head. In an exemplary embodiment, the network setting and detection process are as follows:

[0076] 2.1 First, the input three-channel image is input into the CSPDarknet backbone network. The CSPDarknet backbone network is used to extract features of different sizes of frame images and input the output features of multiple blocks into the feature pyramid. In this embodiment, the intermediate features of the sizes of 64×48×80, 128×24×40, and 256×12×20 output by the 5th, 7th, and 11th block structures of the backbone network are saved as the input of the feature pyramid.

[0077] 2.2 Input the multi-scale features output by CSPDarknet into the feature pyramid structure.

[0078] The role of the feature pyramid is to improve the network's ability to detect objects of different sizes, and to interact with the rich semantic information in low-size feature information and the rich spatial position information in large-size feature information, thereby further improving the accuracy of small target detection.

[0079] For the structure, the feature pyramid includes a first link where low-size features output by multiple blocks propagate to high-size features and a second link where high-size features propagate to low-size features. After the same-size features in the first link and the second link are fused in series, the interactive features are output; for details, refer to Figure 2 , Figure 2In the figure, the feature pyramid propagates from low-scale features (output features of the 11th block of the backbone network) to high-scale features (output features of the 5th block of the backbone network) (i.e., the branch from feature 3 to feature 1 on the right side of the figure), and at the same time, propagates the semantic information of the high-level feature map to the underlying feature map, and performs feature fusion. That is, the feature input is the features output by the 5th, 7th, and 11th block structures of the backbone network, corresponding to the features 123 on the left side of the figure, and the features 123 on the right side are output, and the arrows represent the corresponding calculations of the features.

[0080] In order to further optimize the above technical solution, considering that the vehicle will not appear in the upper 1 / 3 of the image when the actual shooting angle is input, this application crops the output features of the backbone network during inference to ensure that the cropped features still contain some features of the upper 1 / 3 of the picture, avoiding the complete loss of the image information in the upper 1 / 3, while reducing the network calculation amount as much as possible.

[0081] Taking 64*48*80 as an example, 64 represents the length of the channel dimension of the feature, 48 represents the length of the dimension of the feature corresponding to the height of the image, and 80 represents the length of the dimension of the feature corresponding to the width of the image; in this embodiment, the upper third of the features in the second dimension are discarded, that is, the features of rows 1 to 16 are discarded, and the features of rows 17 to 48 are retained. After cropping, the feature scale is 64*32*80;

[0082] At this time, the feature sizes of the feature pyramid output are 64×32×80, 128×16×40, and 256×8×20 respectively.

[0083] 2.3 For the three feature maps of different sizes output by the feature pyramid, prediction is performed through the prediction head, wherein the prediction head includes multiple convolutional layers, each convolutional layer is used to output a vehicle feature map according to an interactive feature. In this embodiment, after matrix deformation and transposition operations, the sizes of the vehicle feature maps output by the three convolutional layers are: 3×32×80×7, 3×16×40×7, and 3×8×20×7, respectively.

[0084] Furthermore, the vehicle feature map includes vehicle detection results for the distant, mid-range and close views. Each vehicle detection result contains 7 channel data, among which the 1st and 2nd digits are the vehicle center coordinate correction coefficients, the 3rd and 4th digits are the vehicle length and width correction coefficients, the 5th and 6th digits are the confidence that the vehicle is a small vehicle or a large vehicle, and the 7th digit is the confidence that the vehicle is a positive example.

[0085] And the correction formula is:

[0086] b x =(2×r x -0.5+g i )×s i

[0087] by =(2×r y -0.5+g i )×s i

[0088]

[0089] In the formula, b x , b y , b h , b w is the center coordinate and vertical and horizontal length of the object predicted by the network, r x 、r y 、r h 、r w is the parameter predicted by the network for regression correction, g i 、s i Represents the size of the preset grid corresponding to this level of feature map and the scaling factor of this level of feature map compared to the original image size, Represents the vertical and horizontal lengths of the preset anchor box, the preset prior parameters of the grid and anchor box in the YOLOv5 algorithm, and is not changed in the present invention.

[0090] 3. Screen the feature graph to obtain a vehicle detection result.

[0091] In this embodiment, the screening process includes:

[0092] A confidence level is set, and the output results of multiple convolutional layers are screened according to the confidence level, and a non-maximum suppression module is used to remove repeated detections of the same vehicle in the preliminary screening results.

[0093] This application effectively reduces the computational complexity of the vehicle detection network through feature clipping, allowing the network to run in real time on an edge computing platform.

[0094] Embodiment 2

[0095] The present application discloses a lane line detection method, comprising:

[0096] The lane line detection network is used to detect the frame image of the monitoring video to obtain the segmentation mask, and the lane line is detected based on the segmentation mask and the lane line instance;

[0097] In this embodiment, after obtaining the vehicle detection result, the lane line is detected using the lane line detection network; wherein the lane line detection network is as follows: Figure 3 As shown, it includes the EfficientNet backbone network, bidirectional feature pyramid and segmentation head;

[0098] The EfficientNet backbone network is used to extract features of different sizes of frame images and input the features output from multiple different stages into the bidirectional feature pyramid;

[0099] Bidirectional feature pyramid, used to fuse features output from multiple different stages;

[0100] The segmentation head is used to perform convolution upsampling and weighted averaging on the fused features and then output the segmentation mask.

[0101] In a specific embodiment, the detection process is as follows:

[0102] 1. Input the downsampled image of size 3×360×640 into the EfficientNet backbone network. The EfficientNet backbone network is used to extract features of different sizes of frame images. In this embodiment, the outputs of the 3rd, 4th, 5th, 6th, and 7th stages of the backbone network are saved. The feature map sizes are 32×96×160, 48×48×80, 136×24×40, 384×12×20, 160×6×10, and 160×3×5, respectively, as the input of the bidirectional feature pyramid and the segmentation head; at the same time, the preliminary extracted features of the second stage are retained to facilitate fusion through the segmentation head;

[0103] 2. Input the output features of the 3rd, 4th, 5th, 6th and 7th stages into the bidirectional feature pyramid. The bidirectional feature pyramid is used to further process the features of different scales and output features of the same size containing multi-scale fusion information.

[0104] Specifically, the bidirectional feature pyramid includes a first path and a second path for propagating high-size features outputted at multiple different stages to low-size features, and features of the same size in the first path and the second path are directly connected, and multiple fused features are sequentially outputted through the second path; and also includes a third path for propagating low-size features in intermediate features to high-size features, and the intermediate features do not include the highest-size features and the lowest-size features in the output features of different stages; the third path is serially connected between corresponding-size features in the first path and the second path, and the lowest-size feature in the third path is integrated into the lowest-size feature in the first path, and the highest-size feature in the third path is propagated to the highest-size feature in the second path;

[0105] In one embodiment, Figure 4As shown in the figure, the output features of the 3rd, 4th, 5th, 6th and 7th stages are input into the bidirectional feature pyramid. The bidirectional feature pyramid realizes bidirectional feature propagation and interaction by introducing the path from low-size features to high-size features (the middle branch from top to bottom) and the propagation path from high-size features to low-size features (the rightmost branch from bottom to top). Specifically, the feature input is the feature output from the 34567th stage of the backbone network, corresponding to the leftmost feature 12345 in the figure, and the rightmost feature 12345 is output, and the arrows represent the corresponding calculation of the features.

[0106] 3. Further, the output features of the second stage are combined with the output features of the second stage through the segmentation head to perform convolution upsampling and weighted averaging on the features of the same size output by the bidirectional feature pyramid containing multi-scale fusion information, and then output the segmentation mask. The output of the second stage in the present invention includes the features of the image preliminarily extracted by the backbone network, and the resolution of the features is high, including more spatial location information and detail information, which plays an important role in pixel-level segmentation.

[0107] In this embodiment, when performing convolution upsampling, the output of the second stage of the backbone network needs to be fused, and the dimensions of the feature channel, height and width obtained are 64, 96 and 160 respectively; further, the above features are weighted averagely and passed through the convolution layer to obtain the output mask, and the mask size is 3×96×160. The three channels represent the confidence scores of the background, the drivable area, and the lane line area respectively.

[0108] To further optimize the above technical solution, after obtaining the segmentation mask, the steps of detecting the lane line based on the segmentation mask and the lane line instance include:

[0109] (1) The area corresponding to the channel with the largest confidence score in the segmentation mask is used as the area category of the current pixel position to obtain the lane line area mask; in this embodiment, the three-channel masks are compared along the channel dimension, and the area represented by the channel with the largest confidence score in each channel is taken as the area category of the pixel position, and the lane line area mask is taken and upsampled to the original image size for subsequent prediction;

[0110] (2) Matching the lane line area mask with the preset lane line instance, and selecting the lane line instance whose points on the preset lane line instance overlap the most with the segmented lane line area as the first lane line in the lane line;

[0111] Obtaining a direction of a second lane line relative to the first lane line in the lane line according to the shooting position; and determining the second lane line;

[0112] In this embodiment, the video shooting angle is determined according to the angle of the lane line to determine whether the other lane line of the construction lane is to the left or right of the current lane line. If the inclination angle between the lane line and the horizontal axis is greater than 90°, it is determined that the other lane line is located on the right side of the current lane line; if the angle is less than 90°, it is determined that the other lane line is located on the left side of the current lane line.

[0113] Further, the second lane line is instantiated according to the first lane line, specifically taking another lane line on the right side of the current lane line as an example, such as Figure 5 As shown, the 7th and 10th sampling points from bottom to top of the segmented area of ​​the current lane line on the vertical axis are taken, and the horizontal coordinate corresponding to another lane line under the same vertical coordinate is searched to the right, and another lane line (straight line) is generated based on the two searched points.

[0114] In a preferred embodiment, the lane line instance is a set of preset lines extending from the bottom edge, and the generation method thereof is as follows: Figure 6 As shown, including:

[0115] Based on the bottom edge of the frame image, equally spaced sampling points are obtained as the starting point of the preset lane line instance.

[0116] Then, starting from these sampling points and starting from a fixed preset angle, a lane line instance is generated every n pixels on the vertical axis. In this application, 20 pixels are preferred.

[0117] The present invention combines pixel-level semantic segmentation with instance-level lane line detection when processing lane lines. On the basis of maintaining the strong generalization of the segmentation algorithm, the instantiated lane line model is introduced to improve the ability of far-end blurred lane lines. In addition, the instantiated lane lines are more conducive to the subsequent positioning algorithm development compared with the pixel-level region segmentation results.

[0118] Embodiment 3

[0119] This embodiment also discloses a vehicle positioning method, such as Figure 7 As shown, the positioning steps include:

[0120] S1. Initialize the vehicle detection network and lane detection network, and import the trained model weights;

[0121] S2, obtaining the current frame image in the monitoring video, detecting the vehicle using the vehicle detection network, and obtaining a vehicle detection result;

[0122] S3: Determine whether it is the first lane line detection.

[0123] If yes, determine whether there is a near-field vehicle in the detected vehicle detection result. If yes, return to S2 to obtain the next frame image and re-detect the vehicle to avoid reducing the lane line segmentation effect due to the occlusion of the near-field vehicle; (Although this application has optimized it, once the occlusion is too serious, it may still affect the effect, so this process needs to be repeated until there is no near-field vehicle)

[0124] If not, perform lane line detection, and determine the lane each vehicle is in based on the lane line detection result and the vehicle detection result;

[0125] If not, determine whether the interval from the last lane line detection reaches the preset frame interval. If not, directly determine the lane where each vehicle is located based on the lane line detection results and the vehicle detection results. If yes, determine whether there is a vehicle in the vehicle detection results. If so, first determine the lane where each vehicle is located based on the lane line detection results and the vehicle detection results. If not, re-detect the lane lines, and then determine the lane where each vehicle is located based on the lane line detection results and the vehicle detection results.

[0126] As an exemplary embodiment, the lane where each vehicle is located is determined according to the lane line detection result and the vehicle detection result, and the process includes:

[0127] The bottom center of the vehicle detection frame in the vehicle detection result is used as the vehicle position in the image.

[0128] Based on the ordinate of the center of the bottom edge, determine the point with the same ordinate on the detected lane line.

[0129] The vehicle is positioned based on the points on the lane line with the same ordinate and the center of the bottom edge.

[0130] That is, the bottom center of the vehicle detection frame is selected as the vehicle position in the image, and the point with the same vertical coordinate on the lane line instance is calculated by the interpolation algorithm through the vertical coordinate of this point. The relative position of the interpolated points and the vehicle on the two instantiated lane lines is determined, and all vehicle targets are traversed to obtain the judgment result of whether there is a vehicle on the construction lane in the current frame.

[0131] It should be noted that the vehicle detection algorithm in the present invention can be any real-time detection algorithm. The lane line detection algorithm can also be replaced by other segmentation networks; however, the vehicle detection algorithm disclosed in the first embodiment and the lane line detection network disclosed in the second embodiment are preferably used.

[0132] Optionally, the vehicle positioning method disclosed in the present application can be applied to highway maintenance construction safety warning; and based on the vehicle positioning result, it is determined whether there is a vehicle located in the construction lane.

[0133] The speed of vehicles on highways is high, and due to the single scene, drivers are prone to dangerous driving situations such as fatigue driving, which leads to major traffic accidents on highways. For road construction safety maintenance scenarios, early warning work is usually completed by radar warning and sound and light information prompts to vehicles. When the precise positioning algorithm of vehicles and lanes provided by this application is applied, auxiliary early warning and comprehensive perception of vehicles and oncoming road conditions can be achieved, which improves the intelligence of early warning and reduces the probability of false alarms and missed alarms.

[0134] The present invention effectively improves the overall efficiency of the algorithm and avoids unnecessary computational overhead by asynchronously processing the lane detection task and the lane detection task. Since the visual sensor has a relatively fixed viewing angle, lane detection does not need to be very frequent. In view of the possible situation where the machine is displaced due to external reasons and the viewing angle changes, re-detection is only performed when the interval is more than a certain frame and there is no vehicle in the scene, so as to improve the overall system robustness.

[0135] In order to verify the positioning effect of the vehicle positioning method of this application, the vehicle detection network and lane detection network were trained using the BDD100K large traffic dataset, where the car class in the training label was classified as a small vehicle, and the rest were classified as large vehicles. It was tested on the Shaanxi Province highway scene test set and achieved good performance.

[0136] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0137] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A vehicle detection method, characterized in that: Detecting the frame images of the monitoring video using a vehicle detection network to obtain a vehicle feature map, and screening the vehicle feature map to obtain a vehicle detection result; The vehicle detection network includes a CSPDarknet backbone network, a feature pyramid and a prediction head connected in sequence; CSPDarknet backbone network, used to extract features of different sizes of frame images and input the output features of multiple blocks into the feature pyramid; Feature pyramid, used to interact the output features of multiple blocks, including a first link for propagating low-size features to high-size features output by multiple blocks and a second link for propagating high-size features to low-size features. After the same-size features in the first link and the second link are fused in series, the interacted features are output; The prediction head includes multiple convolutional layers, each of which is used to output a vehicle feature map based on an interactive feature.

2. A vehicle detection method according to claim 1, characterized in that: The vehicle feature map includes vehicle detection results for distant, mid-range and close views, and each vehicle detection result includes 7 channel data, wherein the first and second digits are vehicle center coordinate correction coefficients, the third and fourth digits are vehicle length and width correction coefficients, the fifth and sixth digits are confidence levels for whether the vehicle is a small vehicle or a large vehicle, and the seventh digit is confidence level for whether the vehicle is a positive example.

3. A vehicle detection method according to claim 2, characterized in that: The vehicle characteristic graph is screened to obtain a vehicle detection result, including: A confidence level is set, and the output results of multiple convolutional layers are screened according to the confidence level, and a non-maximum suppression module is used to remove repeated detections of the same vehicle in the preliminary screening results.

4. A vehicle detection method according to claim 2, characterized in that: The convolution layer modifies the features based on the correction coefficient according to the following formula: b x =(2×r x -0.5+g i )×s i b y =(2×r y -0.5+g i )×s i Where b x 、b y 、b h 、b w is the center coordinate and longitudinal and lateral length of the predicted vehicle, r x 、r y 、r h 、r w are the corresponding correction systems, g i 、s i Represents the size of the preset grid corresponding to this level of feature map and the scaling factor of this level of feature map compared to the original image size, Represents the vertical and horizontal lengths of the preset anchor box.

5. A lane line detection method, characterized in that: The lane line detection network is used to detect the frame image of the monitoring video to obtain the segmentation mask, and the lane line is detected based on the segmentation mask and the lane line instance; The lane detection network includes an EfficientNet backbone network, a bidirectional feature pyramid, and a segmentation head; The EfficientNet backbone network is used to extract features of different sizes of frame images and input the features output from multiple different stages into the bidirectional feature pyramid; A bidirectional feature pyramid is used to fuse features output from multiple different stages, including a first path and a second path for propagating high-size features output from multiple different stages to low-size features, and features of the same size in the first path and the second path are directly connected, and multiple fused features are sequentially output through the second path; it also includes a third path for propagating from low-size features in intermediate features to high-size features, and the intermediate features do not include the highest-size features and the lowest-size features in the output features of different stages; the third path is serially connected between corresponding-size features in the first path and the second path, and the lowest-size feature in the third path is integrated into the lowest-size feature in the first path, and the highest-size feature in the third path is propagated to the highest-size feature in the second path; The segmentation head is used to perform convolution upsampling and weighted averaging on the fused features and then output the segmentation mask.

6. A lane line detection method according to claim 5, characterized in that: Detect lane lines based on segmentation masks and lane line instances, including: The area corresponding to the channel with the largest confidence score in the segmentation mask is used as the area category of the current pixel position to obtain the lane line area mask; The lane line area mask is matched with the preset lane line instance to obtain the first lane line in the lane line; the direction of the second lane line in the lane line relative to the first lane line is obtained according to the shooting position to determine the second lane line.

7. A lane line detection method according to claim 6, characterized in that: The method for generating the preset lane line instance is: Based on the bottom edge of the frame image, get equally spaced sampling points. Taking the sampling point as the starting point and starting from a preset angle, a lane line instance is generated every n pixels on the vertical axis.

8. A vehicle positioning method, characterized in that: The vehicle is positioned based on a vehicle detection network and a lane line detection network, wherein the vehicle detection network is the vehicle detection network in the vehicle detection method described in any one of claims 1 to 4, or the lane line detection network is the lane line detection network in the lane line detection method described in any one of claims 5 to 7, and the positioning step includes: S1, initialize the vehicle detection network and lane line detection network; S2, obtaining the current frame image in the monitoring video, detecting the vehicle using the vehicle detection network, and obtaining a vehicle detection result; S3: Determine whether it is the first lane line detection. If yes, determine whether there is a close-up vehicle in the detected vehicle detection result. If yes, return to S2 to obtain the next frame image and re-detect the vehicle. If no, perform lane line detection and determine the lane where each vehicle is located based on the lane line detection result and the vehicle detection result. If not, determine whether the interval from the last lane line detection reaches the preset frame interval. If not, directly determine the lane where each vehicle is located based on the lane line detection results and the vehicle detection results. If yes, determine whether there is a vehicle in the vehicle detection results. If so, first determine the lane where each vehicle is located based on the lane line detection results and the vehicle detection results. If not, re-detect the lane lines, and then determine the lane where each vehicle is located based on the lane line detection results and the vehicle detection results.

9. A vehicle positioning method according to claim 8, characterized in that: The lane where each vehicle is located is determined based on the lane line detection results and the vehicle detection results. The process includes: The bottom center of the vehicle detection frame in the vehicle detection result is used as the vehicle position in the image. Based on the ordinate of the center of the bottom edge, determine the point with the same ordinate on the detected lane line. The vehicle is positioned based on the points on the lane line with the same ordinate and the center of the bottom edge.

10. A vehicle positioning method according to claim 9, characterized in that: Applied to highway maintenance and construction safety warning; And based on the vehicle positioning results, determine whether there is a vehicle in the construction lane.