Traffic sign missing detection method based on block retrieval strategy

Abstract

The application discloses a traffic signboard missing detection method based on a block retrieval strategy, and comprises the following steps: collecting a video stream on a to-be-detected road section in real time through an image acquisition device; detecting a plurality of traffic signboards in the video stream and obtaining geographical coordinates corresponding to each traffic signboard; determining signboard information of each traffic signboard; constructing an undirected graph and a database according to the geographical coordinates and the signboard information of each traffic signboard; and in the same road section inspection process, comparing information of all traffic signboards in a data block of a corresponding region in the database with the geographical coordinates and the signboard information of the traffic signboards obtained to detect a traffic signboard missing condition. The traffic signboard missing detection method based on the block retrieval strategy naturally associates and maps road information, signboard region division and database construction through the data structure of the undirected graph, and is used for subsequent missing detection.

Description

Technical Field

[0001] This invention belongs to the field of intelligent transportation, specifically relating to a method for detecting missing traffic signs based on a block retrieval strategy. Background Technology

[0002] Road traffic signs are generally placed on both sides of the road and are mainly applicable to highways, urban roads and all special roads. They use graphic symbols and text to convey specific information, manage traffic, indicate driving directions and ensure smooth traffic and driving safety.

[0003] Traffic signs are often damaged or missing due to natural disasters, traffic accidents, and other reasons. Currently, the detection of abnormal traffic signs is mainly carried out by manual inspection. However, due to the large number of roads and the sparse population on highways and expressways, this requires a huge amount of time and manpower, affecting the timeliness of detecting missing traffic signs and updating data, ultimately creating safety hazards for daily traffic. Summary of the Invention

[0004] This invention provides a traffic sign missing detection method based on a block retrieval strategy to solve the aforementioned technical problems, specifically adopting the following technical solution:

[0005] A traffic sign missing detection method based on a block retrieval strategy includes the following steps:

[0006] The video stream on the road section to be inspected is acquired in real time using an image acquisition device;

[0007] Detect several traffic signs in the video stream and obtain the geographic coordinates of each traffic sign;

[0008] Determine the sign information for each traffic sign;

[0009] An undirected graph and database are constructed based on the geographic coordinates and sign information of each traffic sign;

[0010] During the inspection process on the same road segment, the geographical coordinates and sign information corresponding to the traffic signs are compared with all traffic signs in the corresponding data block in the database to detect missing traffic signs.

[0011] Furthermore, the specific method for constructing an undirected graph and database based on the geographic coordinates and sign information of each traffic sign is as follows:

[0012] The traffic signs are divided into several regions based on their geographical coordinates.

[0013] The geographic coordinates and sign information of all traffic signs in the area are combined into data blocks to obtain several data blocks;

[0014] The center coordinates of each area are calculated using the geographic coordinates of all traffic signs and road information within the area;

[0015] An undirected graph is constructed based on the calculated center coordinates of each region. The data blocks corresponding to each region are then indexed and connected in the form of an undirected graph to build a database.

[0016] Furthermore, the specific method for dividing traffic signs into several regions based on their geographical coordinates is as follows:

[0017] The DBSCAN clustering algorithm is used to process the geographic coordinates of traffic signs for regional division.

[0018] Furthermore, the clustering radius is set to 50m, and the clustering threshold is set to 1.

[0019] Furthermore, in the process of calculating the center coordinates of each area using the geographic coordinates and road information of all traffic signs within the area, if there is only one straight line in the area, then the center coordinates of the area are obtained based on the median; if there are intersecting lines in the area, then the intersection point is taken as the center coordinates of the area.

[0020] Furthermore, an undirected graph is constructed based on the calculated center coordinates of each region. The specific method for building the database by indexing and connecting the data blocks corresponding to each region in the form of an undirected graph is as follows:

[0021] Based on road information and center coordinates, an undirected graph with roads as edges and regions as vertices is obtained;

[0022] For any pair of adjacent regions, link their corresponding data blocks according to the undirected edges in the undirected graph to construct a database.

[0023] Furthermore, during the inspection process on the same road segment, the connection between the area and other areas in the undirected graph and the index of the data block are determined based on the direction of movement of the image acquisition device.

[0024] Furthermore, the specific method for detecting several traffic signs in the video stream and obtaining the geographic coordinates corresponding to each traffic sign is as follows:

[0025] Use the YOLOv5 algorithm to detect traffic signs in a video stream;

[0026] The latitude and longitude coordinates of traffic signs are obtained using GPS.

[0027] Furthermore, the specific method for determining the sign information for each traffic sign is as follows:

[0028] The traffic sign information is obtained by classifying traffic sign images using a trained ResNet network model.

[0029] Furthermore, the specific method for acquiring video streams on the road segment to be detected in real time using an image acquisition device is as follows:

[0030] The patrol car uses cameras to collect real-time video streams of the road section to be inspected.

[0031] The advantage of this invention lies in the traffic sign missing detection method based on block retrieval strategy, which makes full use of prior knowledge such as signs, roads and location relationships, and naturally associates and maps road information, sign area division and database construction through an undirected graph data structure, making subsequent missing detection easier. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a flowchart of a traffic sign missing detection method based on a block retrieval strategy according to the present invention;

[0034] Figure 2 This is a diagram illustrating the division of regions;

[0035] Figure 3 This is a diagram illustrating the generation of data blocks;

[0036] Figure 4 This is a schematic diagram of the generated region center;

[0037] Figure 5 It is a schematic diagram of the component undirected graph and data. Detailed Implementation

[0038] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0039] like Figure 1The diagram shows a flowchart of a traffic sign missing detection method based on a block retrieval strategy, which mainly includes the following steps: S1: Real-time acquisition of video streams on the road segment to be detected using an image acquisition device. S2: Detection of several traffic signs in the video stream and acquisition of the geographic coordinates corresponding to each traffic sign. S3: Determination of the sign information for each traffic sign. S4: Construction of an undirected graph and database based on the geographic coordinates and sign information of each traffic sign. S5: In the same road segment inspection process, information comparison is performed between the acquired geographic coordinates and sign information of the traffic signs and all traffic signs in the corresponding data block of the database to detect missing traffic signs. Through the above steps, prior knowledge such as signs, roads, and location relationships is fully utilized. Road information, sign area division, and database construction are naturally associated and mapped through an undirected graph data structure, simplifying the subsequent missing sign detection problem. The following details the above steps.

[0040] For step S1: The video stream on the road segment to be detected is acquired in real time using an image acquisition device.

[0041] In this application, the specific method for acquiring video streams on the road segment to be detected in real time using an image acquisition device is as follows:

[0042] Video streams are captured in real time on the road sections to be inspected via cameras mounted on patrol vehicles. It is also understandable that video recording could be achieved by using drones equipped with cameras.

[0043] For step S2: Detect several traffic signs in the video stream and obtain the geographical coordinates corresponding to each traffic sign.

[0044] In this application, the specific method for detecting several traffic signs in a video stream and obtaining the geographic coordinates corresponding to each traffic sign is as follows:

[0045] The YOLOv5 algorithm was used to detect traffic signs in the video stream. The latitude and longitude coordinates of the traffic signs were obtained using GPS.

[0046] For step S3: Determine the sign information for each traffic sign.

[0047] Specifically, each road traffic sign has the same meaning, such as "Caution: Children," "Speed ​​Limit: 120," and "Slow Down." Data from various road traffic signs is collected and their meanings are used as labels to construct a dataset for model training.

[0048] In this application, the specific method for determining the sign information for each traffic sign is as follows:

[0049] The traffic sign information is obtained by classifying traffic sign images using a trained ResNet network model.

[0050] For step S4: Construct an undirected graph and database based on the geographic coordinates and sign information of each traffic sign.

[0051] After the initial information collection, an undirected graph and database are constructed based on the collected information.

[0052] In this application, the specific method for constructing an undirected graph and database based on the geographic coordinates and sign information of each traffic sign is as follows:

[0053] Traffic signs are divided into several regions based on their geographic coordinates. Specifically, the DBSCAN clustering algorithm is used to process the geographic coordinates of the traffic signs for region division. Clustering is performed based on the relative distance between each pair of coordinates, with a cluster radius of 50m and a clustering threshold of 1 sign. The results are as follows: Figure 2 As shown.

[0054] The geographic coordinates and sign information of all traffic signs in the area are combined into data blocks to obtain several data blocks. The results are as follows: Figure 3 As shown.

[0055] The center coordinates of each area are calculated using the geographic coordinates of all traffic signs and road information within the area. Specifically, in calculating the center coordinates of each area using the geographic coordinates of all traffic signs and road information within the area, if there is only one straight line in the area, the center coordinates of the area are obtained based on the median; if there are intersecting lines in the area, the intersection point is used as the center coordinates of the area. The results are as follows: Figure 4 As shown.

[0056] Understandably, based on the prior knowledge that patrol vehicles travel along the road and that signs exist only within the road area, it can be assumed that the geographical coordinates of all signs within that area form a straight line or an intersection (e.g., at an intersection). If there is only one straight line within the area, the median is used to obtain the center coordinates of the area. If there are intersections within the area, the intersection point is used as the center coordinates of the area.

[0057] An undirected graph is constructed based on the calculated center coordinates of each region. The data blocks corresponding to each region are then indexed and linked in this undirected graph to build the database. Specifically, an undirected graph with roads as edges and regions as vertices is obtained based on road information and center coordinates. For any pair of adjacent regions, their corresponding data blocks are linked according to the undirected edges in the undirected graph to construct the database. The results are as follows: Figure 5As shown.

[0058] For step S5: In the same road segment inspection process, the information of all traffic signs in the corresponding area of ​​the database is compared with the obtained geographical coordinates and sign information of the traffic signs to detect missing traffic signs.

[0059] During patrols, patrol vehicles determine their current location within a defined area based on their own coordinates. If the distance between the patrol vehicle's coordinates and the center coordinates of an area is less than the radius of that area, then the patrol vehicle is within that area.

[0060] During patrols, the connectivity of a given area to other areas in the undirected graph and the indexing of a data block are determined based on the patrol vehicle's direction of travel. In other words, area switching and data block indexing in the database can be automatic during the patrol vehicle's movement, with the patrol vehicle and the area / data block being bound in real-time. This allows for dynamic binding of the corresponding data block in the database to the area being divided during subsequent patrols of the same road segment.

[0061] Each time patrol vehicle sign information is obtained, it is compared within this data block. If a match is found, the sign exists. This process is repeated until the patrolled area is completed. After the patrol is finished, the records in this data block that have not yet been matched represent the missing signs for that area.

[0062] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any way, and all technical solutions obtained by equivalent substitution or equivalent transformation fall within the protection scope of the present invention.

Claims

1. A traffic sign missing detection method based on a block-based retrieval strategy, characterized in that, The method comprises the following steps: Real-time acquisition of a video stream on a to-be-detected road section by an image acquisition device; Detection of a plurality of traffic signs in the video stream and acquisition of geographical coordinates corresponding to each traffic sign; Determination of sign information of each traffic sign; Construction of an undirected graph and a database according to the geographical coordinates and the sign information of each traffic sign; In the same road section inspection process, information comparison is performed between all traffic signs in a data block of a corresponding region in the database and the geographical coordinates and the sign information of the traffic signs acquired to detect a traffic sign missing condition. The specific method for constructing the undirected graph and the database according to the geographical coordinates and the sign information of each traffic sign is: Regional division of the traffic signs according to the geographical coordinates of the traffic signs to obtain a plurality of regions; Grouping of the geographical coordinates and the sign information of all traffic signs in the regions to obtain a plurality of data blocks; Calculation of a center coordinate of each region through the geographical coordinates and road information of all traffic signs in the region; Grouping of the center coordinates of all regions to form an undirected graph, and indexing and connecting of the data blocks corresponding to each region in the form of the undirected graph to construct the database.

2. The traffic sign missing detection method based on the block retrieval strategy according to claim 1, wherein The specific method for regional division of the traffic signs according to the geographical coordinates of the traffic signs is: Regional division through a DBSCAN clustering algorithm processing the geographical coordinates of the traffic signs.

3. The traffic sign missing detection method based on the block retrieval strategy according to claim 2, wherein The aggregation radius is set to 50 m, and the clustering threshold is set to 1.

4. The traffic sign missing detection method based on the block retrieval strategy according to claim 1, wherein In the process of calculating the center coordinate of each region through the geographical coordinates and road information of all traffic signs in the region, if there is only one straight line in the region, the center coordinate of the region is obtained according to the median, and if there are intersection lines in the region, the intersection points are taken as the center coordinates of the region.

5. The traffic sign missing detection method based on the block retrieval strategy according to claim 4, wherein The specific method for grouping the center coordinates of all regions to form an undirected graph, and indexing and connecting the data blocks corresponding to each region in the form of the undirected graph to construct the database is: Obtaining an undirected graph with roads as edges and regions as vertices according to the road information and the center coordinates; For any pair of adjacent regions, the data blocks corresponding to the regions are linked according to the undirected edges corresponding to the regions in the undirected graph to construct the database.

6. The traffic sign missing detection method based on the block retrieval strategy according to claim 5, wherein In the same road section inspection process, the connection of the region with other regions in the undirected graph and the indexing of the data block are determined according to the moving direction of the image acquisition device.

7. The traffic sign missing detection method based on the block retrieval strategy according to claim 1, wherein The specific method for detecting a plurality of traffic signs in the video stream and obtaining geographical coordinates corresponding to each traffic sign is: using a YOLOv5 algorithm to perform target detection on the traffic signs in the video stream; obtaining the longitude and latitude geographical coordinates of the traffic signs through GPS.

8. The traffic sign missing detection method based on the block retrieval strategy according to claim 1, characterized in that, the specific method for determining the sign information of each traffic sign is: performing image classification on the traffic signs through a trained ResNet network model to obtain the sign information corresponding to the traffic signs.

9. The traffic sign missing detection method based on the block retrieval strategy according to claim 1, characterized in that, the specific method for collecting the video stream on the road section to be detected in real time through the image collection device is: collecting the video stream on the road section to be detected in real time through a camera installed on a patrol vehicle.

Images