A traffic sign detection and recognition method based on YOLOv8

Abstract

The application discloses a traffic sign detection and recognition method based on YOLOv8 and particularly relates to the technical fields of computer vision and image processing, and comprises the following specific steps: step one, data collection, wherein the traffic data collected at each traffic intersection is transmitted to a cloud database for storage, and the traffic walking path data collected at each traffic intersection is also transmitted to the cloud database for storage; traffic sign data, traffic walking path data, intersection following vehicle reference data, single-vehicle walking data and intersection traffic light change data are collected in real time by using a camera; the data of each direction detected is compared with the data of the cloud database; the surrounding trees of the intersection following vehicle traffic walking path can be used as a reference object for recognition; the intersection following vehicle, single-vehicle walking data and intersection traffic light change data can also be used as reference objects for automatic recognition; and the detection and recognition of traffic signs are more accurate.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and image processing technology, and more specifically, to a traffic sign detection and recognition method based on YOLOv8. Background Technology

[0002] With the gradual development of autonomous intelligent vehicles, the progress of autonomous intelligent vehicles will inevitably require the rapid development of traffic sign detection and recognition. As a result, intelligent vehicles will soon be ubiquitous. Traffic sign detection and recognition not only accelerates the development of autonomous vehicles, but also plays a very important role in assisting driving in people's daily driving. Therefore, the traffic sign detection and recognition method of YOLOv8 is needed.

[0003] Among the existing published literature, patent publication number CN114998866A discloses a traffic sign recognition method based on an improved YOLOv4. This patent mainly addresses the problem that natural weather, lighting angle, and complex backgrounds of actual roads can affect the recognition accuracy of small target traffic signs, leading to low recognition accuracy. This patent improves upon this by removing the deeper 19×19 prediction layer from the original network, which can shorten the traffic sign detection time. However, this patent still has the following shortcomings in practical use.

[0004] When using this detection and recognition method, even if the prediction layer is changed due to environmental factors such as weather, it can only accurately predict the data. However, the predicted data cannot be self-calibrated based on specific reference objects, which easily leads to errors in the detection data and thus causes recognition errors. Therefore, a traffic sign detection and recognition method based on YOLOv8 is provided. Summary of the Invention

[0005] To overcome the aforementioned shortcomings of the prior art, this invention provides a traffic sign detection and recognition method based on YOLOv8.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A traffic sign detection and recognition method based on YOLOv8, comprising the following specific steps:

[0007] Step 1: Data collection. Traffic data collected from each intersection is sent to the cloud database for storage. Data collected from each traffic path is also sent to the cloud database for storage. Intersection following reference data and single vehicle movement data are stored in the cloud database. Traffic light change data at intersections also need to be input into the cloud database. Finally, a backup copy of these data is stored in the backup cloud database.

[0008] Step 2: Cloud database classification and optimization. Data in the cloud database is updated in an orderly manner according to the update time, with the update time arranged from newest to oldest. The updated old data is backed up to the backup cloud database for storage, while the new data continues to be stored in the cloud database. The new data update capacity is 5-10GB of compressed capacity, and the cloud database is updated once every 3-5 days.

[0009] Step 3: Confirm and organize old data. The updated old data can send a notification to the backend one day later. The backend will trigger an alarm, and the backend staff will need to process the old data within 10-30 minutes. If the old data is determined to be important, it can continue to be stored in the backup cloud database. If the old data is determined to be unimportant, it should be deleted immediately.

[0010] Step 4: Reference object detection. A camera is mounted on the vehicle and collects traffic sign data, traffic walking path data, intersection following reference data, single vehicle walking data, and intersection traffic light change data in real time. The data detected in each direction is compared with the data in the cloud database, and recognition commands are transmitted according to the data in the cloud database.

[0011] Step 5: Reference object identification. During the identification process, trees around the intersection's following traffic path can be used as reference objects for identification. Reference objects can also be automatically identified based on the following vehicles, single vehicle walking data, and traffic light change data at the intersection. When the vehicles ahead are moving and the traffic light change data at the intersection matches, the size of the trees along the walking path and the identification of reference objects can be used as reference objects to identify traffic signal signs.

[0012] Step Six: YOLOv8 Loss Calculation. The traffic sign image recognition loss in YOLOv8 mainly consists of three parts: Classes Loss (using BCE loss, note that only the classification loss of positive samples is calculated), Objectness Loss (using BCE loss), where OBI refers to the CIU between the network-predicted target boundary and the ground truth box; this calculates the OBI loss for all samples), and Location Loss (using CIU loss, note that only the location loss of positive samples is calculated). The calculation method is as follows:

[0013] The main method for calculating the loss is Loss = λ1Lcls + λ2Lobj + λ3Lloc;

[0014] Step 7: Balance the loss calculation at different scales. Three prediction feature layers are used, with different weights applied to the b-loss across these three layers (P3, P4, P5). In the source code, the weight for prediction feature layer (p3) predicting small targets is 4.0, the weight for prediction feature layer (p4) predicting medium targets is 1.0, and the weight for prediction feature layer (p5) predicting large targets is 0.4. These are the hyperparameters set for the COCO dataset, and the calculation method is as follows:

[0015] Lobj=4.0*Lobjsmall+1.0*Lobjmedium+0.4*Lobjlarge;

[0016] Step 8: Eliminate Grid Sensitivity. The collected data needs to be sensitive to certain characteristics. This requires using the offsets o(tx) and σ(ty) of the predicted target center point relative to the top-left corner (Cx, Cy) of the grid. When the actual target center point is very close to the top-left corner (σ(tx) and σ(ty) should approach 0) or the bottom-right corner (σ(tx) and σ(ty) should approach 1), the network's predicted value needs to be negative or positive infinity to be obtained. Such extreme values ​​are generally unattainable by the network. To solve this problem, the offsets are scaled from (0,1) to (-0.5,1.5). This makes it easier for the network's predicted offsets to reach 0 or 1. Therefore, the final formula for calculating the predicted target center point bx,by is:

[0017] bx = (2.σ(tx) - 0.5) + cx

[0018] by=(2.σ(ty)-0.5)+cy, after using the above calculation method, the recognition sensitivity can be checked;

[0019] Step 9: Matching Positive Samples for Detection and Recognition. When matching and recognizing samples, it is necessary to first calculate the aspect ratio between each GT Box and its corresponding Anchor Templates, using the following formula:

[0020] rw = wgt / wat

[0021] rh = hgt / hat;

[0022] Then, calculate the maximum value between these ratios and their reciprocals. This can be understood as calculating the maximum difference between the GT Box and Anchor Templates in the width and height directions, respectively (the ratio is 1 when they are equal, and the difference is minimal), and calculate it according to the following formula:

[0023] rwmax = max(rw, 1 / rw)

[0024] rhmax = max(rh, 1 / rh);

[0025] Next, we calculate the maximum value between rwmax and rhmax, which represents the largest difference between the width and height directions:

[0026] rmax = max(rwmax, rhmax);

[0027] According to the above formula, the main calculation method is to use the fact that the maximum value of the GT Box and the corresponding AnchorTemplate is less than the threshold value of the anchor (which is set to 4.0 by default in the source code). That is, if the height and width ratio of the GT Box and the corresponding AnchorTemplate are not too different, the GT Box will be assigned to the AnchorTemplate. For a certain GT Box, as long as the GT Box is within the range of ×0.25 and ×4.0 of the width and height of a certain AnchorTemplate, it is considered a successful match. This allows for automatic detection and recognition of traffic signs.

[0028] Step 10: Error correction data update. When errors exist in the above data, the error data can be sent to the backend database processing computer. Backend maintenance personnel can see the specific error data, optimize the processing calculation method based on the error data, and continuously revise the error points to reduce the number of errors in traffic sign detection and recognition.

[0029] Preferably, in step one, the traffic walking path data collection tool is mainly a map vehicle equipped with a 3D map collector, and the collected data needs to be uploaded to the cloud database within 5-10 minutes.

[0030] Preferably, in step one, the cloud database has a capacity of 2T-5T, is compatible with the Redis API, and supports three storage media: memory, persistent memory, and ESSD. The backup cloud database has a capacity of 2-5T, and the cloud database communicates with the backup cloud database.

[0031] Preferably, in step two, the backup cloud database stores old data for 2-5 days, and the old data storage capacity is 20GB-100GB.

[0032] Preferably, the prompting time in step three is 10-30 minutes per time, and the prompting time shall not exceed 6-8 times. If the prompting time exceeds 8 times, an alarm shall be triggered, and the number of alarms shall be 1-3.

[0033] Preferably, in step four, the number of cameras is set to 4-8, and the cameras need to be distributed in the front, rear, left and right directions of the vehicle, with 1-2 cameras placed in each direction.

[0034] Preferably, the identification criteria in step five are any two or more of the following vehicle data at the intersection, single vehicle walking data, and traffic light change data at the intersection.

[0035] Preferably, in step six, λ1, λ2, and λ3 are balance coefficients, and each positioning loss calculation needs to be performed 2-3 times within 0.2S-1S, with the average value being taken.

[0036] Preferably, in step seven, P3 is the first prediction layer, P4 is the second prediction layer, and P5 is the third prediction layer. Three layers need to be predicted during the prediction calculation.

[0037] Preferably, in step ten, the optimization process requires 5-8 experts to simultaneously modify the data. During the modification, each expert must sign to confirm whether the modification method is correct, and the modification authority is mainly determined by the majority of experts obeying the minority experts.

[0038] The technical effects and advantages of this invention are as follows:

[0039] 1. This invention uses a camera to collect traffic sign data, traffic walking path data, intersection following reference data, single vehicle walking data, and intersection traffic light change data in real time. The data detected in each direction is compared with the data in the cloud database. During the recognition process, trees around the intersection following traffic walking path can be used as reference objects for recognition. Reference objects can also be automatically identified based on the following vehicles, single vehicle walking data, and intersection traffic light change data. It has the function of reference object-assisted detection and recognition of traffic signs, automatic calibration of detection and recognition data, and better accuracy of traffic sign detection and recognition.

[0040] 2. This invention employs YOLOv8 loss calculation, balances loss calculations at different scales, eliminates grid sensitivity, and matches positive samples for detection and recognition. Various calculation methods address the image interval size for each traffic sign detection and recognition, and statistically calculates the maximum value between these ratios and their reciprocals. This can be understood as calculating the maximum difference between the GT Box and Anchor Templates in the width and height directions, respectively. By optimizing and calculating various recognition error data and detection data, the final data for traffic sign detection and recognition is determined, enabling multi-directional calculation functions and resulting in more accurate recognition and detection data.

[0041] 3. This invention uses data in the cloud database to be updated in an orderly manner according to the update time, with the update time arranged from newest to oldest. The updated old data is backed up and stored in the backup cloud database, while the new data continues to be stored in the cloud database. Old data that is determined to be important can continue to be saved in the backup cloud database, while old data that is determined to be unimportant is deleted. The detection and identification database is constantly updated with road traffic sign data, so the auxiliary reference data is updated faster and the accuracy of the auxiliary identification and detection data is better.

[0042] In summary, through the interaction of the above-mentioned multiple functions, reference objects can be automatically identified based on data on following vehicles at intersections, single-vehicle traffic volume, and traffic light changes at intersections. This provides reference object-assisted detection and recognition of traffic signs. Furthermore, it can optimize and calculate various recognition error data and detection data to determine the final data for traffic sign detection and recognition. Through multi-directional calculations and continuous updates of road traffic sign data using the detection and recognition database, the accuracy of detection and recognition is effectively improved. Attached Figure Description

[0043] Figure 1 This is a schematic diagram illustrating the operation of a traffic sign detection and recognition method based on YOLOv8 according to the present invention.

[0044] Figure 2 This is a table showing the recognition accuracy of three embodiments in a traffic sign detection and recognition method based on YOLOv8 according to the present invention.

[0045] Figure 3 This is a table of identification interval parameters in a traffic sign detection and recognition method based on YOLOv8 according to the present invention. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] Example 1:

[0048] A traffic sign detection and recognition method based on YOLOv8 includes the following specific steps:

[0049] Step 1: Data Collection. The traffic walking path data collection tool mainly consists of a map vehicle equipped with a 3D map data collector. The traffic data collected at each intersection is transmitted to the cloud database within 5 minutes. Then, the traffic walking path data is transmitted to the cloud database for storage. Intersection following reference data and single vehicle walking data are also stored in the cloud database. Traffic light change data at intersections also need to be input into the cloud database. Finally, a backup copy of this data is stored in a backup cloud database with a capacity of 2TB. The cloud database is compatible with the Redis API and supports three storage media: memory, persistent memory, and ESSD. The backup cloud database has a capacity of 2TB and is communicatively connected to the cloud database.

[0050] Step 2: Cloud database classification and optimization. Data in the cloud database is updated in order of update time, with the update time arranged from newest to oldest. The updated old data is backed up and stored in the backup cloud database. The backup cloud database stores the old data for 2 days with a storage capacity of 20GB. New data continues to be stored in the cloud database with a new data update capacity of 5GB compressed. The cloud database is updated once every 3 days.

[0051] Step 3: Confirm and organize old data. Updated old data can send a notification to the backend one day later. The notification time is 10 minutes per notification, and the notification time shall not exceed 6 times. If the notification is more than 8 times, an alarm will be triggered. There is 1 alarm. Backend personnel need to process the old data within 10 minutes. If the old data is determined to be important, it can continue to be stored in the backup cloud database. If the old data is determined to be unimportant, it should be deleted immediately.

[0052] Step 4: Reference Object Detection. Four cameras are mounted on the vehicle and distributed in the front, rear, left, and right directions of the vehicle, with 1-2 cameras in each direction. The cameras collect traffic sign data, traffic walking path data, intersection following reference data, single vehicle walking data, and intersection traffic light change data in real time. The data detected in each direction is compared with the data in the cloud database, and recognition commands are transmitted according to the data in the cloud database.

[0053] Step 5: Reference Object Recognition. During the recognition process, trees around the intersection's following traffic path can be used as reference objects for identification. Alternatively, reference objects can be automatically identified based on the following vehicles, single-vehicle walking data, and traffic light change data at the intersection. When the vehicle ahead is moving and the traffic light change data at the intersection matches, the size of the trees along the walking path and the reference object are used for identification. The identification criteria are any two or more of the following vehicles, single-vehicle walking data, and traffic light change data at the intersection, which can be used as reference objects to identify traffic signal signs.

[0054] Step Six: YOLOv8 Loss Calculation. The traffic sign image recognition loss in YOLOv8 mainly consists of three parts: Classes Loss (using BCE loss, note that only the classification loss of positive samples is calculated), Objectness Loss (using BCE loss), where OBI refers to the CIU between the network-predicted target boundary and the ground truth box; this calculates the OBI loss for all samples), and Location Loss (using CIU loss, note that only the location loss of positive samples is calculated). The calculation method is as follows:

[0055] The main method for calculating the loss is Loss = λ1Lcls + λ2Lobj + λ3Lloc;

[0056] Furthermore, λ1, λ2, and λ3 are balance coefficients, and each positioning loss calculation needs to be performed twice within 0.2 seconds, with the average value being taken.

[0057] Step 7: Balance the loss calculation at different scales. Three prediction feature layers are used, with different weights applied to the b-loss across these three layers (P3, P4, P5). P3 is the first prediction layer, P4 is the second, and P5 is the third. The prediction calculation requires prediction across all three layers. In the source code, the weight for the prediction feature layer (p3) for small targets is 4.0, the weight for the prediction feature layer (p4) for medium targets is 1.0, and the weight for the prediction feature layer (p5) for large targets is 0.4. These are the hyperparameters set for the COCO dataset, and the calculation method is as follows:

[0058] Lobj=4.0*Lobjsmall+1.0*Lobjmedium+0.4*Lobjlarge;

[0059] Step 8: Eliminate Grid Sensitivity. The collected data needs to be sensitive to certain characteristics. This requires using the offsets o(tx) and σ(ty) of the predicted target center point relative to the top-left corner (Cx, Cy) of the grid. When the actual target center point is very close to the top-left corner (σ(tx) and σ(ty) should approach 0) or the bottom-right corner (σ(tx) and σ(ty) should approach 1), the network's predicted value needs to be negative or positive infinity to be obtained. Such extreme values ​​are generally unattainable by the network. To solve this problem, the offsets are scaled from (0,1) to (-0.5,1.5). This makes it easier for the network's predicted offsets to reach 0 or 1. Therefore, the final formula for calculating the predicted target center point bx,by is:

[0060] bx = (2.σ(tx) - 0.5) + cx

[0061] by=(2.σ(ty)-0.5)+cy, after using the above calculation method, the recognition sensitivity can be checked;

[0062] Step 9: Matching Positive Samples for Detection and Recognition. When matching and recognizing samples, it is necessary to first calculate the aspect ratio between each GT Box and its corresponding Anchor Templates, using the following formula:

[0063] rw = wgt / wat

[0064] rh = hgt / hat;

[0065] Then, calculate the maximum value between these ratios and their reciprocals. This can be understood as calculating the maximum difference between the GT Box and Anchor Templates in the width and height directions, respectively (the ratio is 1 when they are equal, and the difference is minimal), and calculate it according to the following formula:

[0066] rwmax = max(rw, 1 / rw)

[0067] rhmax = max(rh, 1 / rh);

[0068] Next, we calculate the maximum value between rwmax and rhmax, which represents the largest difference between the width and height directions:

[0069] rmax = max(rwmax, rhmax);

[0070] According to the above formula, the main calculation method is to use the fact that the maximum value of the GT Box and the corresponding AnchorTemplate is less than the threshold value of the anchor (which is set to 4.0 by default in the source code). That is, if the height and width ratio of the GT Box and the corresponding AnchorTemplate are not too different, the GT Box will be assigned to the AnchorTemplate. For a certain GT Box, as long as the GT Box is within the range of ×0.25 and ×4.0 of the width and height of a certain AnchorTemplate, it is considered a successful match. This allows for automatic detection and recognition of traffic signs.

[0071] Step 10: Error correction and data update. If there are errors in the above data, the erroneous data can be sent to the backend database processing computer. Backend maintenance personnel can see the specific erroneous data and optimize the calculation method based on the erroneous data. By continuously revising the error points, the number of errors in traffic sign detection and recognition can be reduced. Optimization requires 5 experts to modify the data simultaneously. When modifying, all experts must sign to confirm whether the modification method is correct. The main power of modification is that the majority of experts obey the minority experts.

[0072] Example 2:

[0073] A traffic sign detection and recognition method based on YOLOv8 includes the following specific steps:

[0074] Step 1: Data Collection. The traffic walking path data collection tool mainly consists of a map vehicle equipped with a 3D map data collector. The traffic data collected at each intersection is transmitted to the cloud database within 8 minutes. Then, the traffic walking path data is transmitted to the cloud database for storage. Intersection following reference data and single vehicle walking data are also stored in the cloud database. Traffic light change data at intersections also need to be input into the cloud database. Finally, a backup copy of this data is stored in a backup cloud database with a capacity of 3TB. The cloud database is compatible with the Redis API and supports three storage media: memory, persistent memory, and ESSD. The backup cloud database has a capacity of 3TB and is communicatively connected to the cloud database.

[0075] Step 2: Cloud database classification and optimization. Data in the cloud database is updated in order of update time, with the update time arranged from newest to oldest. The updated old data is backed up and stored in the backup cloud database. The backup cloud database stores the old data for 3 days with a storage capacity of 50GB. New data continues to be stored in the cloud database with a new data update capacity of 8GB compressed. The cloud database is updated once every 4 days.

[0076] Step 3: Confirm and organize old data. Updated old data can be prompted to the backend one day later. The prompt time is 20 minutes per prompt, and the prompt should not exceed 7 times. If it exceeds 8 times, an alarm will be triggered. There are 2 alarms. Backend personnel need to process the old data within 20 minutes. If the old data is determined to be important, it can continue to be stored in the backup cloud database. If the old data is determined to be unimportant, it should be deleted immediately.

[0077] Step 4: Reference Object Detection. This involves mounting 4-8 cameras on the vehicle, positioned at the front, rear, left, and right of the vehicle, with one camera at each location. These cameras collect real-time data on traffic signs, traffic movement paths, intersection following reference data, single vehicle movement data, and traffic light changes at intersections. The detected data from each direction is compared with data in the cloud database, and recognition commands are transmitted according to the data in the cloud database.

[0078] Step 5: Reference Object Recognition. During the recognition process, trees around the intersection's following traffic path can be used as reference objects for identification. Alternatively, reference objects can be automatically identified based on the following vehicles, single-vehicle walking data, and traffic light change data at the intersection. When the vehicle ahead is moving and the traffic light change data at the intersection matches, the size of the trees along the walking path and the reference object are used for identification. The identification criteria are any two or more of the following vehicles, single-vehicle walking data, and traffic light change data at the intersection, which can be used as reference objects to identify traffic signal signs.

[0079] Step Six: YOLOv8 Loss Calculation. The traffic sign image recognition loss in YOLOv8 mainly consists of three parts: Classes Loss (using BCE loss, note that only the classification loss of positive samples is calculated), Objectness Loss (using BCE loss), where OBI refers to the CIU between the network-predicted target boundary and the ground truth box; this calculates the OBI loss for all samples), and Location Loss (using CIU loss, note that only the location loss of positive samples is calculated). The calculation method is as follows:

[0080] The main method for calculating the loss is Loss = λ1Lcls + λ2Lobj + λ3Lloc;

[0081] Furthermore, λ1, λ2, and λ3 are balance coefficients, and each positioning loss calculation needs to be performed twice within 0.5 seconds, with the average value being taken.

[0082] Step 7: Balance the loss calculation at different scales. Three prediction feature layers are used, with different weights applied to the b-loss across these three layers (P3, P4, P5). P3 is the first prediction layer, P4 is the second, and P5 is the third. The prediction calculation requires prediction across all three layers. In the source code, the weight for the prediction feature layer (p3) for small targets is 4.0, the weight for the prediction feature layer (p4) for medium targets is 1.0, and the weight for the prediction feature layer (p5) for large targets is 0.4. These are the hyperparameters set for the COCO dataset, and the calculation method is as follows:

[0083] Lobj=4.0*Lobjsmall+1.0*Lobjmedium+0.4*Lobjlarge;

[0084] Step 8: Eliminate Grid Sensitivity. The collected data needs to be sensitive to certain characteristics. This requires using the offsets o(tx) and σ(ty) of the predicted target center point relative to the top-left corner (Cx, Cy) of the grid. When the actual target center point is very close to the top-left corner (σ(tx) and σ(ty) should approach 0) or the bottom-right corner (σ(tx) and σ(ty) should approach 1), the network's predicted value needs to be negative or positive infinity to be obtained. Such extreme values ​​are generally unattainable by the network. To solve this problem, the offsets are scaled from (0,1) to (-0.5,1.5). This makes it easier for the network's predicted offsets to reach 0 or 1. Therefore, the final formula for calculating the predicted target center point bx,by is:

[0085] bx = (2.σ(tx) - 0.5) + cx

[0086] by=(2.σ(ty)-0.5)+cy, after using the above calculation method, the recognition sensitivity can be checked;

[0087] Step 9: Matching Positive Samples for Detection and Recognition. When matching and recognizing samples, it is necessary to first calculate the aspect ratio between each GT Box and its corresponding Anchor Templates, using the following formula:

[0088] rw = wgt / wat

[0089] rh = hgt / hat;

[0090] Then, calculate the maximum value between these ratios and their reciprocals. This can be understood as calculating the maximum difference between the GT Box and Anchor Templates in the width and height directions, respectively (the ratio is 1 when they are equal, and the difference is minimal), and calculate it according to the following formula:

[0091] rwmax = max(rw, 1 / rw)

[0092] rhmax = max(rh, 1 / rh);

[0093] Next, we calculate the maximum value between rwmax and rhmax, which represents the largest difference between the width and height directions:

[0094] rmax = max(rwmax, rhmax);

[0095] According to the above formula, the main calculation method is to use the fact that the maximum value of the GT Box and the corresponding AnchorTemplate is less than the threshold value of the anchor (which is set to 4.0 by default in the source code). That is, if the height and width ratio of the GT Box and the corresponding AnchorTemplate are not too different, the GT Box will be assigned to the AnchorTemplate. For a certain GT Box, as long as the GT Box is within the range of ×0.25 and ×4.0 of the width and height of a certain AnchorTemplate, it is considered a successful match. This allows for automatic detection and recognition of traffic signs.

[0096] Step 10: Error correction and data update. If there are errors in the above data, the erroneous data can be sent to the backend database processing computer. Backend maintenance personnel can see the specific erroneous data and optimize the calculation method based on the erroneous data. By continuously revising the error points, the number of errors in traffic sign detection and recognition can be reduced. Optimization requires 7 experts to modify the data simultaneously. When modifying, all experts must sign to confirm whether the modification method is correct. The main power of modification is that the majority of experts obey the minority experts.

[0097] Example 3:

[0098] A traffic sign detection and recognition method based on YOLOv8 includes the following specific steps:

[0099] Step 1: Data Collection. The traffic walking path data collection tool mainly consists of a map vehicle equipped with a 3D map data collector. The traffic data collected at each intersection is transmitted to the cloud database within 10 minutes. Then, the traffic walking path data is transmitted to the cloud database for storage. Intersection following reference data and single vehicle walking data are also stored in the cloud database. Traffic light change data at intersections also need to be input into the cloud database. Finally, a backup copy of this data is stored in a backup cloud database with a capacity of 5TB. The cloud database is compatible with the Redis API and supports three storage media: memory, persistent memory, and ESSD. The backup cloud database has a capacity of 5TB and is communicatively connected to the cloud database.

[0100] Step 2: Cloud database classification and optimization. Data in the cloud database is updated in order of update time, with the update time arranged from newest to oldest. The updated old data is backed up and stored in the backup cloud database. The backup cloud database stores the old data for 5 days with a storage capacity of 100GB. New data continues to be stored in the cloud database with a new data update capacity of 10GB compressed. The cloud database is updated once every 5 days.

[0101] Step 3: Confirm and organize old data. Updated old data can send a notification to the backend one day later. The notification time is 30 minutes per notification, and the notification time shall not exceed 8 times. If the notification is exceeded 8 times, an alarm will be triggered. There are 3 alarms. Backend personnel need to process the old data within 30 minutes. If the old data is determined to be important, it can continue to be stored in the backup cloud database. If the old data is determined to be unimportant, it should be deleted immediately.

[0102] Step 4: Reference Object Detection. Eight cameras are mounted on the vehicle and distributed in four directions: front, rear, left, and right. Two cameras are placed in each direction. The cameras collect traffic sign data, traffic walking path data, intersection following reference data, single vehicle walking data, and intersection traffic light change data in real time. The data detected in each direction are compared with the data in the cloud database, and recognition commands are transmitted according to the data in the cloud database.

[0103] Step 5: Reference Object Recognition. During the recognition process, trees around the intersection's following traffic path can be used as reference objects for identification. Alternatively, reference objects can be automatically identified based on the following vehicles, single-vehicle walking data, and traffic light change data at the intersection. When the vehicle ahead is moving and the traffic light change data at the intersection matches, the size of the trees along the walking path and the reference object are used for identification. The identification criteria are any two or more of the following vehicles, single-vehicle walking data, and traffic light change data at the intersection, which can be used as reference objects to identify traffic signal signs.

[0104] Step Six: YOLOv8 Loss Calculation. The traffic sign image recognition loss in YOLOv8 mainly consists of three parts: Classes Loss (using BCE loss, note that only the classification loss of positive samples is calculated), Objectness Loss (using BCE loss), where OBI refers to the CIU between the network-predicted target boundary and the ground truth box; this calculates the OBI loss for all samples), and Location Loss (using CIU loss, note that only the location loss of positive samples is calculated). The calculation method is as follows:

[0105] The main method for calculating the loss is Loss = λ1Lcls + λ2Lobj + λ3Lloc;

[0106] Furthermore, λ1, λ2, and λ3 are balance coefficients, and each positioning loss calculation needs to be performed 2-3 times within 0.2S-1S, with the average value being taken.

[0107] Step 7: Balance the loss calculation at different scales. Three prediction feature layers are used, with different weights applied to the b-loss across these three layers (P3, P4, P5). P3 is the first prediction layer, P4 is the second, and P5 is the third. The prediction calculation requires prediction across all three layers. In the source code, the weight for the prediction feature layer (p3) for small targets is 4.0, the weight for the prediction feature layer (p4) for medium targets is 1.0, and the weight for the prediction feature layer (p5) for large targets is 0.4. These are the hyperparameters set for the COCO dataset, and the calculation method is as follows:

[0108] Lobj=4.0*Lobjsmall+1.0*Lobjmedium+0.4*Lobjlarge;

[0109] Step 8: Eliminate Grid Sensitivity. The collected data needs to be sensitive to certain characteristics. This requires using the offsets o(tx) and σ(ty) of the predicted target center point relative to the top-left corner (Cx, Cy) of the grid. When the actual target center point is very close to the top-left corner (σ(tx) and σ(ty) should approach 0) or the bottom-right corner (σ(tx) and σ(ty) should approach 1), the network's predicted value needs to be negative or positive infinity to be obtained. Such extreme values ​​are generally unattainable by the network. To solve this problem, the offsets are scaled from (0,1) to (-0.5,1.5). This makes it easier for the network's predicted offsets to reach 0 or 1. Therefore, the final formula for calculating the predicted target center point bx,by is:

[0110] bx = (2.σ(tx) - 0.5) + cx

[0111] by=(2.σ(ty)-0.5)+cy, after using the above calculation method, the recognition sensitivity can be checked;

[0112] Step 9: Matching Positive Samples for Detection and Recognition. When matching and recognizing samples, it is necessary to first calculate the aspect ratio between each GT Box and its corresponding Anchor Templates, using the following formula:

[0113] rw = wgt / wat

[0114] rh = hgt / hat;

[0115] Then, calculate the maximum value between these ratios and their reciprocals. This can be understood as calculating the maximum difference between the GT Box and Anchor Templates in the width and height directions, respectively (the ratio is 1 when they are equal, and the difference is minimal), and calculate it according to the following formula:

[0116] rwmax = max(rw, 1 / rw)

[0117] rhmax = max(rh, 1 / rh);

[0118] Next, we calculate the maximum value between rwmax and rhmax, which represents the largest difference between the width and height directions:

[0119] rmax = max(rwmax, rhmax);

[0120] According to the above formula, the main calculation method is to use the fact that the maximum value of the GT Box and the corresponding AnchorTemplate is less than the threshold value of the anchor (which is set to 4.0 by default in the source code). That is, if the height and width ratio of the GT Box and the corresponding AnchorTemplate are not too different, the GT Box will be assigned to the AnchorTemplate. For a certain GT Box, as long as the GT Box is within the range of ×0.25 and ×4.0 of the width and height of a certain AnchorTemplate, it is considered a successful match. This allows for automatic detection and recognition of traffic signs.

[0121] Step 10: Error correction and data update. If there are errors in the above data, the erroneous data can be sent to the backend database processing computer. Backend maintenance personnel can see the specific erroneous data and optimize the processing calculation method based on the erroneous data. By continuously revising the error points, the number of errors in traffic sign detection and recognition can be reduced. Optimization requires 8 experts to modify the data simultaneously. When modifying, all experts must sign to confirm whether the modification method is correct. The main power of modification is that the majority of experts obey the minority experts.

[0122] After the above three sets of examples were performed, the parameters obtained during the testing process were compared and identified as follows:

[0123] Based on the data in the above tables, the following table shows the best detection and recognition results;

[0124]

[0125]

[0126] The comparison of the above tables shows that the data in Example 2 is superior, with a rightward shift in detection accuracy, and the ability to identify smaller data points and update the reference database more quickly, resulting in better accuracy in assisted identification and detection data.

[0127] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A traffic sign detection and recognition method based on YOLOv8, characterized in that: The specific steps are as follows: Step 1: Data collection. Traffic data collected from each intersection is sent to the cloud database for storage. Data collected from each traffic path is also sent to the cloud database for storage. Intersection following reference data and single vehicle movement data are stored in the cloud database. Traffic light change data at intersections also need to be input into the cloud database. Finally, a backup copy of these data is stored in the backup cloud database. Step 2: Cloud database classification and optimization. Data in the cloud database is updated in an orderly manner according to the update time, with the update time arranged from newest to oldest. The updated old data is backed up and stored in the backup cloud database, while the new data continues to be stored in the cloud database. The new data update capacity is 5-10GB of compressed capacity, and the cloud database is updated once every 3-5 days. Step 3: Confirm and organize old data. The updated old data will send a notification to the backend one day later. The backend will trigger an alarm, and the backend staff will need to process the old data within 10-30 minutes. If the old data is determined to be important, it can continue to be stored in the backup cloud database. If the old data is determined to be unimportant, it will be deleted immediately. Step 4: Reference object detection. A camera is mounted on the vehicle and collects traffic sign data, traffic walking path data, intersection following reference data, single vehicle walking data, and intersection traffic light change data in real time. The data detected in each direction is compared with the data in the cloud database, and recognition commands are transmitted according to the data in the cloud database. Step 5: Reference object identification. During the identification process, trees around the intersection's following traffic path can be used as reference objects for identification. Reference objects can also be automatically identified based on the following vehicles, single vehicle walking data, and traffic light change data at the intersection. When the vehicles ahead are moving and the traffic light change data at the intersection matches, the size of the trees along the walking path and the identification of reference objects can be used as reference objects to identify traffic signal signs. Step Six: YOLOv8 Loss Calculation. The traffic sign image recognition loss in YOLOv8 mainly consists of three parts: Classes Loss (using BCE loss, note that only the classification loss of positive samples is calculated), Objectness Loss (again using BCE loss, note that OBI here refers to the CIU between the network-predicted target boundary and the GTBox, here the OBI loss of all samples is calculated), and Location Loss (using CIU loss, note that only the location loss of positive samples is calculated). The calculation method is as follows: The main method for calculating the loss is Loss = λ1Lcls + λ2Lobj + λ3Lloc; Step 7: Balance the loss calculation at different scales. Three prediction feature layers are used, with different weights applied to the b-loss in layers P3, P4, and P5. In the source code, the weight for prediction feature layer (p3) for small targets is 4.0, the weight for prediction feature layer (p4) for medium targets is 1.0, and the weight for prediction feature layer (p5) for large targets is 0.

4. These are hyperparameters set for the COCO dataset, and the calculation method is as follows: Lobj=4.0*Lobjsmall+1.0*Lobjmedium+0.4*Lobjlarge; Step 8: Eliminate Grid Sensitivity. The collected data needs to be sensitive to identification issues. This requires using the offset of the predicted target center point relative to the top-left corner (Cx, Cy) of the grid, denoted as o(tx) and σ(ty). When the actual target center point is very close to the top-left corner of the grid (σ(tx) and σ(ty) should approach 0), or to the bottom-right corner (σ(tx) and σ(ty) should approach 1), the network's predicted value needs to be negative infinity or positive infinity to be obtained. Such extreme values ​​are generally unattainable by the network. To solve this problem, the offset is scaled from (0, 1) to (-0.5, 1.5). This makes it easier for the network's predicted offset to reach 0 or 1. Therefore, the final formula for calculating the predicted target center point bx, by is: bx = (2.σ(tx) - 0.5) + cx by = (2.σ(ty) - 0.5) + cy. After using the above calculation method, the recognition sensitivity can be checked. Step 9: Matching Positive Samples for Detection and Recognition. When matching and recognizing samples, it is necessary to first calculate the aspect ratio between each GT Box and its corresponding Anchor Templates, using the following formula: rw=wgt / wat rh=hgt / hat; Then, calculate the maximum value between these ratios and their reciprocals. This can be understood as calculating the maximum difference between the GT Box and AnchorTemplates in the width and height directions, respectively. When they are equal, the ratio is 1, and the difference is minimal. And calculate according to the following formula: rwmax=max(rw,1 / rw) rhmax = max(rh, 1 / rh); Next, we calculate the maximum value between rwmax and rhmax, which represents the largest difference between the width and height directions: rmax = max(rwmax, rhmax); According to the above formula, the main factor to consider during calculation is the ratio of the height and width of the GT Box and the corresponding Anchor Template. The maximum value of rmax is less than the threshold value of the anchor template. In the source code, the default value of the anchor template is 4.

0. This means that if the height and width ratio of the GT Box and the corresponding Anchor Template are not too different, the GT Box will be assigned to the Anchor Template. For a given GT Box, as long as the GT Box is within the range of 0.25 times the width and height of a given Anchor Template and 4.0 times the height, it is considered a successful match. This allows for automatic detection and recognition of traffic signs. Step 10: Error correction data update. When errors exist in the above data, the error data can be sent to the backend database processing computer. Backend maintenance personnel can see the specific error data, optimize the processing calculation method based on the error data, and continuously revise the error points to reduce the number of errors in traffic sign detection and recognition.

2. The traffic sign detection and recognition method based on YOLOv8 according to claim 1, characterized in that: In step one, the traffic walking path data collection tool is mainly a map vehicle equipped with a 3D map data collector. The collected data needs to be uploaded to the cloud database within 5-10 minutes.

3. The traffic sign detection and recognition method based on YOLOv8 according to claim 1, characterized in that: In step one, the cloud database has a capacity of 2T-5T and is compatible with the Redis API, supporting three storage media: memory, persistent memory, and ESSD. The backup cloud database has a capacity of 2-5T and is connected to the cloud database.

4. The traffic sign detection and recognition method based on YOLOv8 according to claim 1, characterized in that: In step two, the backup cloud database stores old data for 2-5 days, with a storage capacity of 20GB-100GB.

5. The traffic sign detection and recognition method based on YOLOv8 according to claim 1, characterized in that: In step three, the prompting time is 10-30 minutes per time, and the prompting time shall not exceed 6-8 times. If the prompting time exceeds 8 times, an alarm shall be triggered, and the number of alarms shall be 1-3.

6. The traffic sign detection and recognition method based on YOLOv8 according to claim 1, characterized in that: In step four, the number of cameras is set to 4-8, and the cameras need to be distributed in the front, rear, left and right directions of the vehicle, with 1-2 cameras placed in each direction.

7. The traffic sign detection and recognition method based on YOLOv8 according to claim 1, characterized in that: The identification criteria in step five are any two or more of the following vehicle data at the intersection, single vehicle walking data, and traffic light change data at the intersection.

8. The traffic sign detection and recognition method based on YOLOv8 according to claim 1, characterized in that: In step six, λ1, λ2, and λ3 are balance coefficients, and each positioning loss calculation needs to be performed 2-3 times within 0.2S-1S, with the average value being taken.

9. The traffic sign detection and recognition method based on YOLOv8 according to claim 1, characterized in that: In step seven, P3 is the first prediction layer, P4 is the second prediction layer, and P5 is the third prediction layer. Three layers need to be predicted during the prediction calculation.

10. The traffic sign detection and recognition method based on YOLOv8 according to claim 1, characterized in that: In step ten, optimization requires 5-8 experts to modify the data simultaneously. Each expert must sign off on the modification to confirm its correctness, and the majority of experts must yield to the minority experts in making the modification.

Images