Video-based road sign anomaly detection method, system, and storage medium

Abstract

The application discloses a video-based road sign anomaly detection method and system and a storage medium. The method comprises the following steps: acquiring a driving video of a road to be detected, and identifying the driving video to obtain identified road signs; acquiring a first detection time interval and a second detection time interval corresponding to all the identified road signs, and determining whether there is a road sign spacing anomaly according to whether the second detection time interval corresponding to a previous road sign and the first detection time interval corresponding to a subsequent road sign in two adjacent road signs overlap. The detection method can detect road sign anomalies that may occur during driving, including detecting whether the road sign distance is appropriate, thereby improving the real-time performance of road sign anomaly detection and helping to standardize road sign facilities.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a video-based road sign anomaly detection method, system, and storage medium. Background Art

[0002] As a kind of road traffic infrastructure, road signs provide information support for the standardized travel of vehicles and pedestrians, and play an indispensable role in the normal operation of the entire transportation system. Therefore, complete road signs are an important guarantee for the safe and orderly operation of the transportation system.

[0003] In addition to manual detection, existing road sign detection is mostly based on computer vision technology to detect and identify road signs. However, existing detection methods cannot detect road sign anomalies that may occur during certain vehicle driving processes. For example, during driving, due to unreasonable road sign position or distance settings, the time from the latest road sign entering the driver's field of view plus the safety reaction time overlaps with the time when the next road sign enters the field of view, thereby affecting the driver's judgment. Such road sign anomalies that may occur during actual driving cannot be ignored, but existing detection methods have not yet covered such road sign anomalies that may occur during driving. Summary of the Invention

[0004] The purpose of this application is to provide a video-based road sign anomaly detection method, system and storage medium, which can detect road sign anomalies that may occur during driving and further strengthen the standardization of road sign facilities.

[0005] In a first aspect, the present application provides a video-based road sign anomaly detection method using the following technical solutions:

[0006] A method for detecting anomalies of road signs based on video, comprising the following steps:

[0007] Obtain driving video of the road to be detected;

[0008] Identifying the driving video and obtaining identified road signs;

[0009] Obtaining first detection time intervals corresponding to all recognized road signs, and sorting the road signs and the corresponding first detection time intervals in recognition time order to form a set;

[0010] Adding a preset safety time to a first detection time interval corresponding to the identified road sign to obtain a corresponding second detection time interval, wherein the preset safety time includes a preset reaction time and a preset control time;

[0011] define a road sign corresponding to an earlier first detection time interval of two adjacent first detection time intervals in the detection set as a first sign, and a road sign corresponding to a later first detection time interval as a second sign;

[0012] determine whether the second detection time interval of the first sign and the first detection time interval of the second sign overlap;

[0013] if the second detection time interval of the first sign and the first detection time interval of the second sign overlap, output first prompt information and overlap information, the first prompt information indicating that there is a road sign spacing anomaly, and the overlap information including overlapping time and corresponding two road signs.

[0014] According to the above technical solution, the driving speed during driving, the safety reaction time, and the acquisition time of road sign information can be comprehensively judged to determine whether there is a road sign spacing anomaly, so as to enhance the guarantee of road driving safety.

[0015] Optionally, the first detection time interval corresponding to each identified road sign is obtained, including:

[0016] frame-by-frame identification is performed on the driving video, if the previous frame has no road sign and the current frame has a road sign, the current frame is recorded as a start frame; if the previous frame has a road sign and the current frame has no road sign, the current frame is recorded as an end frame;

[0017] The time period between the start frame and the corresponding end frame is sequentially intercepted as a visible time segment to obtain a plurality of visible time segments.

[0018] adjacent visible time segments are sequentially determined whether they correspond to the same road sign,

[0019] if the adjacent visible time segments correspond to the same road sign, the adjacent visible time segments are combined into a new visible time segment;

[0020] if the adjacent visible time segments do not correspond to the same road sign, the previous visible time segment is taken as a continuous visible time.

[0021] According to the continuous visible time, the start time and the end time are respectively obtained to form a first detection time interval.

[0022] Optionally, the determination of whether the adjacent visible time segments correspond to the same road sign includes:

[0023] determine whether the road signs corresponding to the adjacent two visible time segments are the same,

[0024] if not, the adjacent visible time segments do not correspond to the same road sign.

[0025] If they are the same, determine whether the time interval between two adjacent visible time segments is less than the preset time interval.

[0026] If so, the adjacent visible time segments correspond to the same road sign;

[0027] If not, the adjacent visible time segments do not correspond to the same road sign.

[0028] Optionally, obtaining the start time according to the continuous visibility time includes:

[0029] The first recorded time frame during the continuous visibility period is recorded as the start visibility time;

[0030] According to the preset speed record, obtain the driving speed corresponding to the continuous visibility time;

[0031] Obtain a first estimated distance of a time frame corresponding to the start visibility time;

[0032] Determine whether the first estimated distance is within a preset road sign earliest visible distance range,

[0033] If so, the visibility start time is used as the start time of the first detection time interval;

[0034] If not, calculating a first obstruction distance based on the first estimated distance and a preset earliest visible distance range of the road sign;

[0035] Obtaining a first blocking time according to the first blocking distance and the driving speed;

[0036] The new time is obtained by subtracting the first occlusion time from the start visibility time, and is used as the start time of the first detection time interval.

[0037] Optionally, obtaining the end time based on the continuous visibility time includes:

[0038] The last recorded time frame in the continuous visible time is recorded as the end visible time;

[0039] According to the preset speed record, obtain the driving speed corresponding to the continuous visibility time;

[0040] Obtain a second estimated distance of a time frame corresponding to the end visibility time;

[0041] determining whether the second estimated distance is within a preset road sign latest visible distance range;

[0042] If so, the end visibility time is used as the end time of the first detection time interval;

[0043] If not, calculating a second obstruction distance based on the second estimated distance and a preset road sign latest visible distance range;

[0044] Obtaining a second blocking time according to the second blocking distance and the driving speed;

[0045] The end visibility time is added to the second occlusion time to obtain a new time, which is used as the end time of the first detection time interval.

[0046] Optionally, after obtaining the first detection time interval corresponding to all recognized road signs, the method further includes:

[0047] Obtaining the longest continuous visible time represented by the first detection time interval, and defining the time represented by the continuous visible time as the representative time;

[0048] determining whether the representative time is less than a preset reaction time, wherein the preset reaction time represents the time for observing and receiving the road sign information;

[0049] If the representative time is less than the preset reaction time, a second prompt message is output, where the second prompt message is used to indicate that there is an abnormality in the road sign driving recognition.

[0050] Optionally, if the second detection time interval of the first mark overlaps with the first detection time interval of the second mark, after outputting the first prompt information and the overlap information, the method further includes:

[0051] According to the overlapping information, obtain the overlapping time and the corresponding two road signs;

[0052] According to the preset speed record, obtain the driving speed corresponding to the overlapping time period;

[0053] The abnormal offset distance is calculated based on the overlapping time and the corresponding driving speed.

[0054] Optionally, before obtaining the first detection time interval corresponding to all recognized road signs, the method further includes:

[0055] Match the identified road sign information with the pre-built benchmark road sign library; determine whether there is any detection loss based on the matching results;

[0056] If detection is missing, a third prompt message is output, where the third prompt message is used to indicate that road sign detection is missing.

[0057] In a second aspect, the present application provides a video-based road sign anomaly detection system, comprising:

[0058] An acquisition module is used to acquire driving videos of the road to be detected;

[0059] A recognition module is used to recognize driving videos and obtain recognized road signs;

[0060] a detection module, configured to obtain first detection time intervals corresponding to all recognized road signs, and sort the road signs and the corresponding first detection time intervals in order of recognition time to form a set;

[0061] Adding a preset safety time to a first detection time interval corresponding to the identified road sign to obtain a corresponding second detection time interval, wherein the preset safety time includes a preset reaction time and a preset control time;

[0062] a judgment module, configured to define a road sign corresponding to an earlier first detection time interval of two adjacent first detection times in a detection set as a first sign, and a road sign corresponding to a later first detection time interval as a second sign;

[0063] determining whether the second detection time interval of the first mark overlaps with the first detection time interval of the second mark;

[0064] If the second detection time interval of the first sign overlaps with the first detection time interval of the second sign, first prompt information and overlap information are output, wherein the first prompt information is used to indicate that abnormal road sign spacing exists, and the overlap information includes the overlap time and the corresponding two road signs.

[0065] In a third aspect, the present application provides a computer-readable storage medium storing a computer program that can be loaded by a processor and execute the above-mentioned video-based road sign anomaly detection method.

[0066] In summary, the present application can comprehensively judge whether there is an abnormality in the spacing between road signs based on the driving speed, safety reaction time and the time for obtaining road sign information during driving. In terms of the time for obtaining road signs, it will take into account whether there are situations such as occlusion that interfere with the acquisition of road sign information, ensuring that the detection data is more real and effective. At the same time, it will also detect the abnormality of unrecognized road signs during driving and the abnormality of the continuous visibility time of road signs being less than the safety reaction time, thereby improving the real-time performance of road sign detection and helping to further strengthen the standardization of road sign facilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 is a flow chart of an embodiment of the present application;

[0068] Figure 2 This is a flowchart of obtaining the first detection time interval corresponding to all identified road signs in an embodiment of the present application;

[0069] Figure 3 Schematic diagram of a video-based road sign anomaly detection system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] The following is combined with Figure 1 -Attached Figure 3 , further details of this application are given.

[0071] This application provides a method for detecting road sign anomalies based on video. Figure 1 , including the following steps:

[0072] S100: Obtain a driving video of a road to be detected.

[0073] In the embodiment of the present application, the driving video is a video obtained by a detection person driving a vehicle normally on the road to be detected using a camera device equipped with the same viewing angle as that of the human eye.

[0074] S200: Identify the driving video and obtain identified road signs.

[0075] In an embodiment of the present application, in order to reduce the amount of data input and simplify the calculation process, the driving video can be segmented according to preset time periods, and then the segmented video can be identified. The key frame images in the segmented video are detected and identified using a preset target detection algorithm, and the obtained recognition results are the predicted road signs and confidence levels corresponding to the detected targets.

[0076] Among them, the key frame image is the image frame containing complete road signs. In order to obtain the key frame more quickly and conveniently, the PyAV library in Python is directly introduced to decode the driving video of the road to be detected and extract the key frames.

[0077] Among them, the preset target detection algorithm is a road sign detection model obtained by training the existing road sign data set through a machine learning algorithm. By passing the road sign detection model to all key frame images, the detection and recognition results and the corresponding confidence levels can be obtained. The confidence level here is the classification confidence level, which indicates the probability that the detection target is a certain road sign, and the effective detection and recognition results will be screened out based on the confidence score and the preset threshold. That is to say, when the confidence score is too low, it is considered to be an invalid detection. Therefore, the road sign recognition results finally obtained are all valid recognition results after screening. At the same time, for multiple recognition results of the same target, the one with the highest confidence score will be used as the representative recognition result.

[0078] S300 : Obtain corresponding first detection time intervals for all recognized road signs, and sort the road signs and the corresponding first detection time intervals in order of recognition time to form a set.

[0079] The first detection time interval represents the time from when the road sign first completely enters the field of vision to when the road sign first begins to leave the field of vision, because when the road sign is not completely presented in the field of vision, it is difficult to capture the complete information conveyed by the road sign.

[0080] In the embodiment of the present application, the first detection time interval corresponding to each identified road sign is obtained, see Figure 2 , specifically including the following steps:

[0081] S310 , performing frame-by-frame recognition on the driving video. If there is no road sign in the previous frame but there is a road sign in the current frame, the current frame is recorded as the start frame; if there is a road sign in the previous frame but there is no road sign in the current frame, the current frame is recorded as the end frame.

[0082] The start frame represents the time frame when a road sign is first recognized during frame-by-frame recognition of a driving video, and the end frame represents the time frame when a road sign recognized in the previous start frame cannot be recognized for the first time.

[0083] In the embodiment of the present application, since the visibility time of a road sign may be blocked by other vehicles or reflected by other vehicles during driving, the visibility time of the road sign may not be continuous and may appear as multiple visible time segments. Therefore, it is necessary to integrate the visible time segments corresponding to these road signs in order to better obtain the first detection time interval corresponding to the road sign.

[0084] S320 , sequentially intercepting time periods between a start frame and a corresponding end frame as visible time segments to obtain multiple visible time segments.

[0085] In an embodiment of the present application, after obtaining the start frame and end frame corresponding to a road sign, it is necessary to record the time period between the start frame and the end frame as a visible time segment of the road sign. The visible time segment indicates that the road sign is always visible within the time period of the visible time segment.

[0086] S330: Determine in sequence whether adjacent visible time segments correspond to the same road sign.

[0087] S340: If the adjacent visible time segments correspond to the same road sign, combine the adjacent visible time segments into a new visible time segment.

[0088] S350: If adjacent visible time segments do not correspond to the same road sign, use the previous visible time segment as the continuous visible time.

[0089] In an embodiment of the present application, after obtaining a visible time segment of a road sign, the driving video continues to be recognized frame by frame. After obtaining the next visible time segment, if there is occlusion during driving, it is possible that the visible time of the road sign will have multiple discontinuous visible time periods. Therefore, it is necessary to determine whether the road sign corresponding to the next visible time segment is the road sign.

[0090] If so, it means that the next visible time segment still belongs to the visible time segment of the road sign. At this time, the next visible time segment is combined with the previous visible time segment to form a new visible time segment.

[0091] If not, it means that the next visible time segment corresponds to another road sign, which means that the road sign has obtained all the visible time segments up to this point. Therefore, the previous visible time segment is used as the continuous visible time of the road sign. The continuous visible time here represents a set containing one or more visible time segments.

[0092] In the embodiment of the present application, determining whether adjacent visible time segments correspond to the same road sign specifically includes the following steps:

[0093] S331. Determine whether the road signs corresponding to two adjacent visible time segments are the same.

[0094] S332: If they are not the same, the adjacent visible time segments do not correspond to the same road sign.

[0095] S333: If they are the same, determine whether the time interval between two adjacent visible time segments is less than a preset time interval.

[0096] S334: If yes, the adjacent visible time segments correspond to the same road sign.

[0097] S335: If not, the adjacent visible time segments do not correspond to the same road sign.

[0098] Among them, to determine whether the road signs corresponding to two adjacent visible time segments are the same, the judgment can be made based on the road sign recognition results corresponding to the two visible time segments. If the road signs represented by the recognition results are the same, it means that the two road signs are the same. The similarity can also be calculated based on the road sign recognition results corresponding to the two visible time segments. When the similarity between the two reaches a set threshold, it can also be said that the two road signs are the same.

[0099] In the embodiment of the present application, the judgment is made directly based on the road sign recognition results corresponding to the two visible time segments. If the road signs represented by the recognition results are the same, it means that the road signs corresponding to the two visible time segments are the same.

[0100] In the embodiment of the present application, since there may be situations where road signs are identical, when two adjacent road signs are identical, if only determining whether the road signs are identical is performed, when obtaining the visible time segments corresponding to the two road signs, it will be impossible to determine which specific road sign's visible time segment it is, resulting in inaccurate acquisition of the continuous visible time of the road signs.

[0101] Therefore, in order to further confirm that the road signs corresponding to the two visible time segments are the same road sign, it is also necessary to determine whether the time interval between the two visible time segments is less than the preset time interval. Because during driving, even if the same road sign is recognized continuously, there is a certain time interval. Therefore, the preset time interval is used for further limitation. If the road signs corresponding to the two visible time segments are the same, and the time interval between the two visible time segments is less than the preset time interval, it means that the road signs corresponding to the two visible time segments are the same road sign.

[0102] S360: Obtain a start time and an end time according to the continuous visibility time to form a first detection time interval.

[0103] The start time indicates the earliest time when the road sign completely enters the sight, and the end time indicates the earliest time when the road sign begins to disappear from the sight.

[0104] In the embodiment of the present application, due to the existence of occlusion, lighting and other situations during driving, in addition to causing the visibility time of the road signs to be discontinuous, it may also cause the time when the road signs begin to enter the field of vision to be delayed, and the time when the road signs leave the field of vision to be advanced. For the latter two situations, that is, occlusion in special circumstances, the beginning occlusion and the end occlusion, the time when the road signs should be visible is shortened. Therefore, it is necessary to repair the shortened time part, that is, the occlusion time, so that the subsequent abnormality judgment data can be more accurate, more real and effective.

[0105] In the embodiment of the present application, obtaining the start time according to the continuous visibility time specifically includes the following steps:

[0106] S3601. Record the first recorded time frame in the continuous visible time as the start visible time.

[0107] The start visible time refers to the time when a road sign is first recognized.

[0108] That is, the first recorded time frame of the continuous visibility time corresponding to the road sign.

[0109] S3602. Obtain the driving speed corresponding to the continuous visibility time according to the preset speed record.

[0110] In an embodiment of the present application, the preset speed record is to record the driving speed corresponding to any time through an external speed recording device. Therefore, the preset speed record can be used to obtain the driving speed corresponding to each visible time segment in the continuous visible time, and finally the average value is taken as the driving speed corresponding to the continuous visible time.

[0111] S3603: Obtain a first estimated distance of a time frame corresponding to the start visibility time.

[0112] In the embodiment of the present application, the first estimated distance represents the distance relative to the road sign in the time frame of the start visibility time.

[0113] S3604: Determine whether the first estimated distance is within a preset road sign earliest visible distance range.

[0114] The preset earliest visible time range of the road sign indicates that the vehicle can begin to capture the road sign information when the distance between the vehicle and the road sign is within the earliest visible distance range.

[0115] In the implementation of this application, the preset earliest road sign visibility time range is an estimated value range that comprehensively considers factors such as vehicle size, human vision, weather, etc. By determining whether the first estimated distance is within the preset earliest road sign visibility distance range, it can be determined whether there is any obstruction.

[0116] S3605: If yes, use the start visibility time as the start time of the first detection time interval.

[0117] In an embodiment of the present application, if the first estimated distance is within the preset earliest visible distance range of the road sign, it means that the road sign was recognized within the time period when the road sign should be visible, that is, a normal recognition without any obstruction. Therefore, the start visibility time can be used as the start time of the corresponding road sign, that is, the earliest time when the road sign comes into sight.

[0118] S3606: If not, calculate a first blocking distance based on the first estimated distance and a preset earliest visible distance range of the road sign.

[0119] In the embodiment of the present application, if the first estimated distance is not within the preset earliest visible distance range of the road sign, it indicates that the road sign is not identified in the time period when the road sign should be visible, that is, there is an obstruction, so that the earliest time to identify the road sign is delayed. The first estimated distance and the preset earliest visible distance range of the road sign can be used to estimate the distance of the obstruction, that is, the first obstruction distance.

[0120] S3607, according to the first obstruction distance and the driving speed, the first obstruction time is obtained.

[0121] In the embodiment of the present application, after the distance of the obstruction is obtained, the obstruction time, that is, the first obstruction time, can be calculated according to the current driving speed.

[0122] S3608, the start visible time is subtracted by the first obstruction time to obtain a new time, and the new time is taken as the start time of the first detection time interval.

[0123] In the embodiment of the present application, after the obstruction time is calculated, that is, the time delayed to identify the road sign due to the obstruction, the new time obtained by delaying the start visible time by the first obstruction time is taken as the start time of the first detection time, that is, the earliest time when the road sign starts to enter the line of sight.

[0124] In the embodiment of the present application, the end time is obtained according to the continuous visible time, and specifically includes the following steps:

[0125] S3611, the last recorded time frame in the continuous visible time is recorded as the end visible time.

[0126] The end visible time represents the last time when the road sign is identified

[0127] That is, the last recorded time frame in the continuous visible time corresponding to the road sign.

[0128] S3612, the driving speed corresponding to the continuous visible time is obtained according to the preset speed record.

[0129] In the embodiment of the present application, the preset speed record is recorded by the speed recording device of the external device at any time, so that the driving speed corresponding to each visible time segment in the continuous visible time can be obtained through the preset speed record, and the average value is finally taken as the driving speed corresponding to the continuous visible time.

[0130] S3613, the second estimated distance of the time frame corresponding to the end visible time is obtained.

[0131] In the embodiment of the present application, the second estimated distance represents the distance relative to the road sign in the time frame of the end visibility time.

[0132] S3614: Determine whether the second estimated distance is within a preset road sign maximum visible distance range.

[0133] The preset latest visible time range for road signs indicates that when the distance between the vehicle and the road sign is within the latest visible distance range, the road sign information can still be captured, that is, the road sign is at a distance where it is about to disappear from sight.

[0134] In the implementation of this application, the preset road sign latest visible time range is an estimated value range that comprehensively considers factors such as vehicle size, human vision, weather, etc. By determining whether the second estimated distance is within the preset road sign latest visible distance range, it can be determined whether there is any obstruction.

[0135] S3615: If yes, use the end visibility time as the end time of the first detection time interval.

[0136] In an embodiment of the present application, if the second estimated distance is within the preset road sign's latest visible distance range, it means that the road sign was identified during the latest time period when the road sign can be seen, that is, a normal recognition without any obstruction. Therefore, the end visibility time can be used as the end time of the corresponding road sign, that is, the earliest time when the road sign begins to disappear from sight.

[0137] S3616: If not, calculate a second blocking distance based on the second estimated distance and a preset road sign latest visible distance range.

[0138] In an embodiment of the present application, if the second estimated distance is not within the preset range of the earliest visible distance of the road sign, it means that the road sign is not recognized during the latest time period when the road sign can be seen, that is, there is occlusion, which makes the latest time when the road sign can be recognized advanced. The distance of the occlusion, that is, the second occlusion distance, can be estimated through the second estimated distance and the preset range of the latest visible distance of the road sign.

[0139] S3617. Obtain a second blocking time according to the second blocking distance and the driving speed.

[0140] In the embodiment of the present application, after obtaining the blocked distance, the blocking time, that is, the second blocking time, can be calculated based on the current driving speed.

[0141] S3618: Add the second obstruction time to the end visibility time to obtain a new time, and use it as the end time of the first detection time interval.

[0142] In an embodiment of the present application, after the occlusion time is calculated, that is, the latest time at which the road sign can be recognized is advanced due to the occlusion, the new time obtained by postponing the end visibility time by the second occlusion time is used as the end time of the first detection time, that is, the time when the road sign first begins to disappear from sight.

[0143] S400: Add a preset safety time to a first detection time interval corresponding to the recognized road sign to obtain a corresponding second detection time interval.

[0144] Among them, the preset safety time includes a preset reaction time and a preset control time. The preset reaction time represents the time from capturing a road sign to receiving road sign information, and the preset control time represents the time for the driver to take corresponding response measures based on the obtained road sign information.

[0145] In this embodiment of the present application, the vehicle is in motion from the time the latest road sign information is received to the time the corresponding response is made. During this time, the next road sign may come into view, which may affect the driver's judgment. Therefore, the detection time interval of adjacent road signs is used to detect such anomalies. The second detection time interval here represents the period formed by adding the preset safety time to the end time of the first detection time interval.

[0146] S500: Determine whether the second detection time interval of the first mark overlaps with the first detection time interval of the second mark.

[0147] In the embodiment of the present application, the road sign corresponding to the earlier first detection time interval of two adjacent first detection time intervals in the detection set is defined as the first sign, and the road sign corresponding to the later first detection time interval is defined as the second sign.

[0148] After obtaining the identified road signs, the road signs and the corresponding first detection time intervals and second detection time intervals will be sorted according to the time sequence of identification to form a set. Since the first detection time intervals and second detection time intervals corresponding to two adjacent road signs need to be compared, the comparison objects will be pre-defined. All road signs in the set will be grouped into two adjacent road signs. The first road sign in each group is defined as the first sign, and the second road sign is defined as the second sign. Each group will be compared and judged in turn to obtain the detection results.

[0149] S600: If the second detection time interval of the first mark overlaps with the first detection time interval of the second mark, output first prompt information and overlap information.

[0150] The judgment of whether the second detection time interval of the first sign and the first detection time interval of the second sign overlap is also the judgment of whether there is an intersection between the second detection time interval of the first road sign and the first detection time interval of the first road sign. If there is an intersection, it means that there is an overlap. The first prompt information is used to indicate that there is a road sign spacing anomaly. The overlap information includes the overlap time and the corresponding two road signs.

[0151] In the embodiment of the present application, by taking two adjacent road signs as a group of detection objects, the first sign and the second sign in each group are compared and judged in turn. If the second detection time interval of the first sign in a certain group overlaps with the first detection time interval of the second sign, it means that there is an anomaly. Therefore, the first prompt information is output to indicate that there is a road sign spacing anomaly. The overlap time is obtained, and the overlap time and the corresponding two road signs are fed back as overlap information.

[0152] In the embodiment of the present application, after obtaining the first detection time interval corresponding to all the identified road signs, the following steps are further included:

[0153] S710, obtaining the longest continuous visible time in the first detection time interval, and defining the time represented by the continuous visible time as a representative time.

[0154] In the embodiment of the present application, during driving, situations such as line of sight obstruction and reflection may occur, which may reduce the time during which the complete information of the road sign can be captured. That is, the obtained first detection time interval may include one or more continuous visible time periods. Each continuous visible time period represents that the road sign is visible. However, there may be a situation that the continuous visible time period is too short, even lower than the preset reaction time, that is, there is not enough time to observe and receive the road sign information. In this case, it is also easy to cause safety hazards.

[0155] Therefore, by comparing the longest continuous visible time in the first detection time interval corresponding to the road sign and the reaction time, the specific reason why the road sign cannot be identified during the road sign visible time period during driving can be further analyzed, so as to analyze and infer whether there is a road sign anomaly.

[0156] S720, judging whether the representative time is less than a preset reaction time.

[0157] The preset reaction time represents the time of observing and receiving the road sign information.

[0158] S730: If the representative time is less than the preset reaction time, output a second prompt message, where the second prompt message is used to indicate that there is an abnormality in the road sign driving recognition.

[0159] The representative time represents the longest continuous visibility time in the first detection time interval. If the longest continuous visibility time is less than the preset reaction time, it indicates that there is an abnormality in road sign recognition during the driving process. Therefore, a prompt message is output and the corresponding road sign and the first detection time interval are recorded for further specific analysis of the abnormality.

[0160] In an embodiment of the present application, if the second detection time interval of the first mark overlaps with the first detection time interval of the second mark, after outputting the first prompt information and the overlap information, the following steps are further included:

[0161] S810: Obtain overlapping time and two corresponding road signs according to the overlapping information.

[0162] In an embodiment of the present application, when it is determined that the second detection time interval of the first sign of two adjacent road signs overlaps with the first detection time interval of the second sign, an overlapping information pair will be fed back and output, and the overlapping information includes the overlapping time and the corresponding two road signs.

[0163] S820: Obtain the driving speed corresponding to the overlapping time period according to the preset speed record.

[0164] In the embodiment of the present application, the driving speed is obtained by an external speed recording device, and the driving speed in the overlapping time period is obtained from the external speed recording device. , which is the average driving speed during the overlapping time periods.

[0165] S830: Calculate and obtain the abnormal offset distance based on the overlapping time and the corresponding driving speed.

[0166] In the embodiment of the present application, the abnormal offset distance represents the difference between the distance between the abnormal road signs and the safe and effective distance. and the corresponding driving speed of the road signs , you can calculate the abnormal offset distance , specifically expressed as .

[0167] In the embodiment of the present application, before obtaining the first detection time interval corresponding to all recognized road signs, the following steps are also included:

[0168] S910: Match the recognized road sign information with a pre-built reference road sign library.

[0169] The reference road sign library is a distribution information library of real road signs of the to-be-detected road. By matching and comparing the reference road sign library with the recognized road sign information, it can be determined whether there is a road sign that cannot be recognized.

[0170] S920, determining whether detection loss occurs according to the matching result.

[0171] S930, if the detection loss occurs, outputting third prompt information, wherein the third prompt information is used to indicate that there is road sign detection loss.

[0172] The third information is used to indicate that there is road sign detection loss. In the embodiment of the application, by matching and comparing the recognized road sign information and the reference road sign library, it can be determined whether there is a road sign that cannot be recognized. If there is a road sign that cannot be recognized, the third prompt information is input, and the road sign information that cannot be recognized is output as feedback.

[0173] The embodiment of the application also provides a video-based road sign anomaly detection system, which is shown in Figure 3 The system comprises an acquisition module 101, an identification module 102, a detection module 103, and a judgment module 104.

[0174] The acquisition module 101 is used to acquire a driving video of a to-be-detected road.

[0175] The identification module 102 is used to identify the driving video and acquire recognized road signs.

[0176] The detection module 103 is used to acquire, for all the recognized road signs, a first detection time interval corresponding to each road sign, and sort the road signs and the corresponding first detection time intervals in a time order to form a set.

[0177] The first detection time interval corresponding to the recognized road sign is added with a preset safety time to acquire a corresponding second detection time interval.

[0178] The judgment module 104 is used to define a road sign corresponding to an earlier first detection time interval of two adjacent first detection time intervals in the detection set as a first sign, and a road sign corresponding to a later first detection time interval as a second sign.

[0179] It is judged whether the second detection time interval of the first sign and the first detection time interval of the second sign overlap.

[0180] If the second detection time interval of the first sign and the first detection time interval of the second sign overlap, the first prompt information and the overlapping information are output.

[0181] In an embodiment of the present application, the acquisition module is specifically used to obtain a driving video of the road to be detected, identify the driving video, and detect road sign anomalies that may occur during the driving process based on the recognition results and the corresponding time frame.

[0182] In an embodiment of the present application, the recognition module is specifically used to recognize driving videos by segmenting the videos and extracting key frames, and obtaining detection and recognition results for the key frame images according to a preset target detection algorithm.

[0183] In an embodiment of the present application, the detection module is specifically used to determine a first detection time interval for all identified road signs, that is, the earliest time and the latest time when the road sign is identified. At the same time, based on the safety response time for receiving road sign information and taking corresponding response measures, a second detection time interval is formed, and abnormality detection is performed through the first detection time interval and the second detection time interval of adjacent road signs.

[0184] In an embodiment of the present application, the judgment module is specifically used to detect all identified road signs, with two adjacent road signs as a group of detection objects, and define the previous road sign in each group as the first sign and the next road sign as the second sign. The second detection time interval of the first sign of each group and the next first detection time interval of the second sign are compared in turn to determine if there is any overlap. If overlap occurs, a first prompt message is output to indicate that there is a road sign abnormality, and the overlapping information is fed back and output.

[0185] An embodiment of the present application further provides a computer-readable storage medium storing a computer program that can be loaded by a processor and execute any of the above-mentioned video-based road sign anomaly detection methods.

[0186] The embodiments of this specific implementation method are all preferred embodiments of the present application and are not intended to limit the scope of protection of the present application. Therefore, all equivalent changes made based on the principles of the present application should be included in the scope of protection of the present application.

Claims

1. A video-based road sign anomaly detection method, characterized in that: include: Obtain driving video of the road to be detected; Identifying the driving video and obtaining identified road signs; Obtaining a first detection time interval corresponding to each identified road sign, the first detection time interval being a continuous time period between the earliest time the road sign completely enters the field of view and the earliest time it begins to disappear from the field of view; the first detection time interval represents the time from the earliest time the road sign completely enters the field of view to the earliest time it begins to disappear from the field of view; and sorting the road signs and their corresponding first detection time intervals in order of identification time to form a set; Adding a preset safety time to a first detection time interval corresponding to the identified road sign to obtain a corresponding second detection time interval, wherein the preset safety time includes a preset reaction time and a preset control time; Define the road sign corresponding to the earlier first detection time interval of two adjacent first detection time intervals in the detection set as the first sign, and the road sign corresponding to the later first detection time interval as the second sign; determining whether the second detection time interval of the first mark overlaps with the first detection time interval of the second mark; If the second detection time interval of the first sign overlaps with the first detection time interval of the second sign, first prompt information and overlap information are output, wherein the first prompt information is used to indicate that abnormal road sign spacing exists, and the overlap information includes the overlap time and the corresponding two road signs.

2. The method for detecting road sign anomalies based on video according to claim 1, characterized in that: The step of obtaining first detection time intervals corresponding to all recognized road signs includes: Perform frame-by-frame recognition on the driving video. If there is no road sign in the previous frame but there is a road sign in the current frame, the current frame is recorded as the start frame; if there is a road sign in the previous frame but there is no road sign in the current frame, the current frame is recorded as the end frame. Sequentially intercepting the time period between the start frame and the corresponding end frame as a visible time segment to obtain multiple visible time segments; Determine in sequence whether adjacent visible time segments correspond to the same road sign. If adjacent visible time segments correspond to the same road sign, the adjacent visible time segments are combined into a new visible time segment; If adjacent visible time segments do not correspond to the same road sign, the previous visible time segment is regarded as the continuous visible time; According to the continuous visibility time, a start time and an end time are respectively obtained to form a first detection time interval.

3. The method for detecting road sign anomalies based on video according to claim 2, characterized in that: The determining whether adjacent visible time segments correspond to the same road sign includes: Determine whether the road signs corresponding to two adjacent visible time segments are the same. If they are not the same, the adjacent visible time segments do not correspond to the same road sign; If they are the same, determine whether the time interval between two adjacent visible time segments is less than the preset time interval. If so, the adjacent visible time segments correspond to the same road sign; If not, the adjacent visible time segments do not correspond to the same road sign.

4. The method for detecting road sign anomalies based on video according to claim 2, characterized in that: The step of obtaining the start time according to the continuous visible time includes: The first recorded time frame in the continuous visibility time is recorded as the start visibility time; and the driving speed corresponding to the continuous visibility time is obtained according to the preset speed record; Obtain a first estimated distance of a time frame corresponding to the start visibility time; determining whether the first estimated distance is within a preset earliest visible distance range of the road sign; If so, the visibility start time is used as the start time of the first detection time interval; If not, calculating a first obstruction distance based on the first estimated distance and a preset earliest visible distance range of the road sign; Obtaining a first blocking time according to the first blocking distance and the driving speed; The new time is obtained by subtracting the first occlusion time from the start visibility time, and is used as the start time of the first detection time interval.

5. The method for detecting road sign anomalies based on video according to claim 2, characterized in that: The step of obtaining the end time according to the continuous visible time includes: The last recorded time frame in the continuous visible time is recorded as the end visible time; According to the preset speed record, obtain the driving speed corresponding to the continuous visibility time; Obtain a second estimated distance of a time frame corresponding to the end visibility time; determining whether the second estimated distance is within a preset road sign latest visible distance range; If so, the end visibility time is used as the end time of the first detection time interval; If not, calculating a second obstruction distance based on the second estimated distance and a preset road sign latest visible distance range; Obtaining a second blocking time according to the second blocking distance and the driving speed; The end visibility time is added to the second occlusion time to obtain a new time, which is used as the end time of the first detection time interval.

6. The method for detecting road sign anomalies based on video according to claim 3, characterized in that: After obtaining the first detection time interval corresponding to all the recognized road signs, the method further includes: Obtaining the longest continuous visible time represented by the first detection time interval, and defining the time represented by the continuous visible time as the representative time; Determining whether the representative time is less than a preset reaction time, where the preset reaction time represents the time it takes to observe and receive the road sign information; If the representative time is less than the preset reaction time, a second prompt message is output, where the second prompt message is used to indicate that there is an abnormality in the road sign driving recognition.

7. The method for detecting road sign anomalies based on video according to claim 1, characterized in that: If the second detection time interval of the first mark overlaps with the first detection time interval of the second mark, after outputting the first prompt information and the overlap information, the method further includes: According to the overlapping information, the overlapping time and the corresponding two road signs are obtained; According to the preset speed record, obtain the driving speed corresponding to the overlapping time period; The abnormal offset distance is calculated based on the overlapping time and the corresponding driving speed.

8. The method for detecting road sign anomalies based on video according to claim 1, characterized in that: Before acquiring the first detection time interval corresponding to all the recognized road signs, the method further includes: Match the identified road sign information with the pre-built benchmark road sign library; determine whether there is any detection loss based on the matching results; If detection is missing, a third prompt message is output, where the third prompt message is used to indicate that road sign detection is missing.

9. A video-based road sign anomaly detection system comprising: An acquisition module is used to acquire driving videos of the road to be detected; A recognition module is used to recognize driving videos and obtain recognized road signs; a detection module, configured to obtain first detection time intervals corresponding to all recognized road signs, and sort the road signs and the corresponding first detection time intervals in order of recognition time to form a set; Adding a preset safety time to a first detection time interval corresponding to the identified road sign to obtain a corresponding second detection time interval, wherein the preset safety time includes a preset reaction time and a preset control time; a judgment module, configured to define a road sign corresponding to an earlier first detection time interval of two adjacent first detection times in a detection set as a first sign, and a road sign corresponding to a later first detection time interval as a second sign; determining whether the second detection time interval of the first mark overlaps with the first detection time interval of the second mark; If the second detection time interval of the first sign overlaps with the first detection time interval of the second sign, first prompt information and overlap information are output, wherein the first prompt information is used to indicate that abnormal road sign spacing exists, and the overlap information includes the overlap time and the corresponding two road signs.

10. A computer-readable storage medium storing a computer program capable of being loaded by a processor and executing the video-based road sign anomaly detection method according to any one of claims 1 to 8.

Images