Traffic marking detection method and system based on target detection and background modeling
Through the method based on object detection and background modeling, the YOLOv5 network and rotary object detection algorithm are used to solve real-time and missed detection problems in traffic marking detection, achieving more efficient marking positioning and type recognition, avoiding point clouds The use of data improves detection efficiency and accuracy.
Patent Information
- Application Number
- CN202411065380.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-05
- Publication Date
- 2024-11-29
AI Technical Summary
The existing traffic marking detection methods have shortcomings in real-time and missed detection issues, especially in the failure to effectively deal with the situation where markings are blocked by vehicles, and relying on time-consuming point cloud data and segmentation algorithms lead to inefficiency.
Using a method based on object detection and background modeling, the vehicle target is detected through multiple pictures, the road background picture is modeled, and the YOLOv5 network and rotation object detection algorithm are used to locate the location and type of traffic markings, avoiding the use of point cloud data. , reducing computational complexity.
It improves the real-time and accuracy of traffic marking detection, reduces missed detection, reduces the amount of calculation, improves detection efficiency, and is faster than the segmentation algorithm.
Smart Images

Figure CN119049011A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image recognition, and in particular to a traffic marking detection method and system based on target detection and background modeling. Background Art
[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.
[0003] Traffic markings are signs such as arrows and text drawn on the ground to control and guide traffic. Traffic road marking detection is to identify the meaning of arrows and text, and its recognition results can be applied to autonomous driving and traffic violation review.
[0004] The patent with publication number CN116665169A discloses an online detection method for traffic markings, which requires original laser point cloud data of the current frame. Processing the point cloud data is an extremely time-consuming operation, so its real-time performance is poor.
[0005] The patent with publication number CN116434176A discloses a traffic marking detection method, which extracts features from the acquired image based on a lightweight convolutional neural network, and segments the feature fusion image based on a segmentation algorithm to obtain a temporary detection result of the traffic marking. However, it does not consider the situation where the marking is blocked by a vehicle, which can easily lead to missed detection. Summary of the invention
[0006] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a traffic marking detection method and system based on target detection and background modeling. The present invention can effectively solve the problem of missed detection by using multiple pictures to detect vehicle targets and modeling the background pictures of the road.
[0007] In order to achieve the above object, the present invention adopts the following technical solution:
[0008] A first aspect of the present invention provides a traffic marking detection method based on target detection and background modeling.
[0009] A traffic marking detection method based on target detection and background modeling, comprising:
[0010] Get several pictures containing the same traffic marking;
[0011] Use the target detection algorithm to detect the location of the vehicle in each picture;
[0012] Calculate the vehicle area of each image, select the image with the smallest vehicle area as the benchmark image, traverse the remaining images, calculate the non-maximum suppression of the vehicle area of the benchmark image and the vehicle areas of other images, and crop the vehicle area of the image with the smallest non-maximum suppression to the benchmark image; repeat this step until all vehicle targets of the benchmark image are traversed, and obtain the modeled road background image;
[0013] The modeled road background image is input into the traffic marking detection network to obtain the location and type of traffic markings.
[0014] Furthermore, the target detection algorithm adopts the YOLOv5 network.
[0015] Furthermore, the YOLOv5 network includes: a backbone network for extracting image features, a neck network for fusing features and outputting feature maps of different scales, and a head network for predicting the category and position of the target.
[0016] Furthermore, a rotating target detection algorithm is used to train the traffic marking detection network.
[0017] A second aspect of the present invention provides a traffic marking detection system based on target detection and background modeling.
[0018] A traffic marking detection system based on target detection and background modeling, comprising:
[0019] The image acquisition module is configured to: acquire a plurality of images containing the same traffic marking;
[0020] A vehicle detection module is configured to: detect the position of the vehicle in each image using a target detection algorithm;
[0021] The background subtraction modeling module is configured to: calculate the vehicle area of each image, select the image with the smallest vehicle area as the benchmark image, traverse the remaining images, calculate the non-maximum suppression of the vehicle area of the benchmark image and the vehicle areas of other images, and crop the vehicle area of the image with the smallest non-maximum suppression to the benchmark image; repeat this step until all vehicle targets of the benchmark image are traversed to obtain a modeled road background image;
[0022] The traffic marking detection module is configured to: input the modeled road background image into the traffic marking detection network to obtain the position and type of the traffic marking.
[0023] Furthermore, the target detection algorithm adopts the YOLOv5 network.
[0024] Furthermore, the YOLOv5 network includes: a backbone network for extracting image features, a neck network for fusing features and outputting feature maps of different scales, and a head network for predicting the category and position of the target.
[0025] Furthermore, a rotating target detection algorithm is used to train the traffic marking detection network.
[0026] A third aspect of the present invention provides a computer-readable storage medium.
[0027] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the traffic marking detection method based on target detection and background modeling as described in the first aspect above.
[0028] A fourth aspect of the present invention provides a computer device.
[0029] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the traffic marking detection method based on target detection and background modeling as described in the first aspect above are implemented.
[0030] Compared with the prior art, the present invention has the following beneficial effects:
[0031] The current detection of some traffic markings requires the use of point cloud data and segmentation algorithms, both of which are very time-consuming. The present invention does not require the use of point cloud data, and only needs to use a target detection algorithm to accurately locate the position and type of the markings, which is faster than the segmentation algorithm.
[0032] Most of the current traffic marking detections do not take marking occlusion into consideration, which may cause missed detections. The present invention uses multiple images to detect vehicle targets and model the background images of the road, which can effectively solve the missed detection problem.
[0033] The present invention can optimize the real-time performance required for traffic marking detection, does not use point cloud data with complex calculations and a segmentation network with a larger number of model parameters, reduces the amount of calculations, and improves the efficiency of traffic marking detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.
[0035] Figure 1 It is a flow chart of a traffic marking detection method based on target detection and background modeling shown in the present invention. DETAILED DESCRIPTION
[0036] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.
[0037] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.
[0038] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.
[0039] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods and systems according to various embodiments of the present disclosure. It should be noted that each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of a code may include one or more executable instructions for implementing the logical functions specified in each embodiment. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the flowchart and / or block diagram, and the combination of boxes in the flowchart and / or block diagram can be implemented using a dedicated hardware-based system that performs a specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.
[0040] Embodiment 1
[0041] like Figure 1As shown, this embodiment provides a traffic marking detection method based on target detection and background modeling. This embodiment uses the method applied to a server as an example for illustration. It is understandable that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communications, middleware services, domain name services, security services CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in this application. In this embodiment, the method includes the following steps:
[0042] S1: reads multiple images containing traffic markings captured by the front end;
[0043] S2: Use the target detection algorithm to detect the position of the vehicle in each picture and retain the results. The vehicle detection algorithm uses the mature YOLOv5 network, which consists of three parts: backbone, neck and head. The specific structure diagram is as follows: the backbone is responsible for extracting the features of the picture, the neck is responsible for fusing the features and outputting feature maps of different scales, and the head is responsible for predicting the category and position of the target.
[0044] S3: Calculate the sum of vehicle areas of multiple images, select the image with the smallest vehicle area as the benchmark image, traverse the remaining images, calculate the non-maximum suppression (nms) of the vehicle area of the benchmark image and the vehicle area of other images, and crop the area of the image with the smallest nms to the benchmark image. Repeat this step until all vehicle targets of the benchmark image are traversed. At this time, the vehicle target of the benchmark image has been replaced by the area of the same position in other images that is not blocked by vehicles, completing the background modeling under the front-end camera.
[0045] Specifically, the process of calculating the non-maximum suppression (nms) of the vehicle area of the reference image and the vehicle area of other images includes: assuming that the reference image has N vehicles, cyclically calculating the nms of the vehicle of the reference image and all vehicles of other images, and saving them into an array. After the loop is completed, the image corresponding to the minimum value in the nms array is obtained, and the coordinate area corresponding to the vehicle of the reference image is replaced with the corresponding vehicle area of the corresponding image.
[0046] Among them, non-maximum suppression is the ratio of the intersection of two rectangular boxes to the union of the two matrix boxes.
[0047] S4: The modeled road background image is input into the trained traffic marking detection network. Due to the diversity of camera angles, the present invention uses a rotating target detection algorithm for training. Rotating target detection adds an angle offset based on vehicle target detection, which can better fit the original shape of the object and avoid occlusion.
[0048] Specifically, the process of training the traffic marking detection network using the rotating target detection algorithm includes: the dataset is annotated. The rotating target box is usually represented by five parameters: center point coordinates (x, y), width (w), height (h) and rotation angle (θ). The data is then input into the backbone to extract features such as texture and color of the image. The FPN network is then used to obtain and fuse features of different scales. The head is responsible for classifying each anchor box or candidate area to determine whether the area contains the target object and its category. The detection head also needs to accurately regress the bounding box coordinates of the object.
[0049] In this embodiment, the traffic marking detection network can use the YOLO5-obb network to detect the traffic markings by rotating the rectangular frame.
[0050] The current detection of some traffic markings requires the use of point cloud data and segmentation algorithms, both of which are very time-consuming. The present invention does not require the use of point cloud data, and only needs to use a target detection algorithm to accurately locate the position and type of the markings, which is faster than the segmentation algorithm.
[0051] S5: Output the position and type of traffic markings under this front camera.
[0052] In order to transmit the results of traffic marking detection to downstream applications in real time, the present invention uses a simpler data source RGB picture and a lighter network structure.
[0053] Embodiment 2
[0054] This embodiment provides a traffic marking detection system based on target detection and background modeling.
[0055] A traffic marking detection system based on target detection and background modeling, comprising:
[0056] The image acquisition module is configured to: acquire a plurality of images containing the same traffic marking;
[0057] A vehicle detection module is configured to: detect the position of the vehicle in each image using a target detection algorithm;
[0058] The background subtraction modeling module is configured to: calculate the vehicle area of each image, select the image with the smallest vehicle area as the benchmark image, traverse the remaining images, calculate the non-maximum suppression of the vehicle area of the benchmark image and the vehicle areas of other images, and crop the vehicle area of the image with the smallest non-maximum suppression to the benchmark image; repeat this step until all vehicle targets of the benchmark image are traversed to obtain a modeled road background image;
[0059] The traffic marking detection module is configured to: input the modeled road background image into the traffic marking detection network to obtain the position and type of the traffic marking.
[0060] In some embodiments, the target detection algorithm uses a YOLOv5 network.
[0061] In some embodiments, the YOLOv5 network includes: a backbone network for extracting image features, a neck network for fusing features and outputting feature maps of different scales, and a head network for predicting the category and position of the target.
[0062] In some embodiments, a rotating target detection algorithm is used to train the traffic marking detection network.
[0063] Embodiment 3
[0064] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the traffic marking detection method based on target detection and background modeling as described in the first embodiment above are implemented.
[0065] Embodiment 4
[0066] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the traffic marking detection method based on target detection and background modeling as described in the first embodiment above are implemented.
[0067] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.
[0068] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0069] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0070] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0071] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.
[0072] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A traffic marking detection method based on target detection and background modeling, characterized in that: include: Get several pictures containing the same traffic marking; Use the target detection algorithm to detect the location of the vehicle in each picture; Calculate the vehicle area of each image, select the image with the smallest vehicle area as the benchmark image, traverse the remaining images, calculate the non-maximum suppression of the vehicle area of the benchmark image and the vehicle areas of other images, and crop the vehicle area of the image with the smallest non-maximum suppression to the benchmark image; Repeat this step until all vehicle targets in the benchmark image are traversed to obtain the modeled road background image; The modeled road background image is input into the traffic marking detection network to obtain the location and type of traffic markings.
2. The traffic marking detection method based on target detection and background modeling according to claim 1, characterized in that: The target detection algorithm adopts the YOLOv5 network.
3. The traffic marking detection method based on target detection and background modeling according to claim 2 is characterized in that: The YOLOv5 network includes: a backbone network for extracting image features, a neck network for fusing features and outputting feature maps of different scales, and a head network for predicting the category and position of the target.
4. The traffic marking detection method based on target detection and background modeling according to claim 1, characterized in that: The traffic marking detection network is trained using a rotating target detection algorithm.
5. A traffic marking detection system based on target detection and background modeling, characterized in that: include: The image acquisition module is configured to: acquire a plurality of images containing the same traffic marking; A vehicle detection module is configured to: detect the position of the vehicle in each image using a target detection algorithm; A background subtraction modeling module is configured to: calculate the vehicle area of each picture, select the picture with the smallest vehicle area as the reference picture, traverse the remaining pictures, calculate the non-maximum suppression of the vehicle area of the reference picture and the vehicle areas of other pictures, and crop the vehicle area of the picture with the smallest non-maximum suppression to the reference picture; Repeat this step until all vehicle targets in the benchmark image are traversed to obtain the modeled road background image; The traffic marking detection module is configured to: input the modeled road background image into the traffic marking detection network to obtain the position and type of the traffic marking.
6. The traffic marking detection system based on target detection and background modeling according to claim 5, characterized in that: The target detection algorithm adopts the YOLOv5 network.
7. The traffic marking detection system based on target detection and background modeling according to claim 6, characterized in that: The YOLOv5 network includes: a backbone network for extracting image features, a neck network for fusing features and outputting feature maps of different scales, and a head network for predicting the category and position of the target.
8. The traffic marking detection system based on target detection and background modeling according to claim 5, characterized in that: The traffic marking detection network is trained using a rotating target detection algorithm.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the traffic marking detection method based on target detection and background modeling as described in any one of claims 1 to 4 are implemented.
10. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the traffic marking detection method based on target detection and background modeling as described in any one of claims 1 to 4 are implemented.
Citation Information
Patent Citations
Lane guide arrow identification method in intersection monitoring environment
CN111476157A
Road traffic marking identification method and device
CN114495060A
Background modeling method and device and computer storage medium
CN115937241A
