Complex multi-target hierarchical and joint precise detection method for smart transportation

Abstract

The present invention proposes a complex multi-target hierarchical joint precision detection method for intelligent transportation. First, three types of target detection models for single-dimensional scenes are constructed. Then, the corresponding single-dimensional time detection model and single-dimensional weather detection model are selected according to the current time class and weather class. These two models are then combined with m3 single-dimensional target detection models for hierarchical joint detection. Finally, a hierarchical joint mechanism is used as the standard for hierarchical joint detection. For the first level of joint detection, the target category is determined in turn according to the two different levels of dimension and probability; for the second level of joint detection, the target category is determined in turn according to the two different levels of compliance and probability. The invention can be widely used in the field of machine vision for complex multi-target hierarchical joint precision detection of transportation. While ensuring the cost, it can achieve all-day and all-weather precision detection of various targets, and has a very broad application prospect.

Description

Technical Field

[0001] The present invention relates to a complex multi-target hierarchical and graded joint precise detection method for smart transportation, and belongs to the field of machine vision. Background Art

[0002] In recent years, with the rapid development of deep learning and the emergence of high-performance graphics cards, the development of computer vision technology has been greatly promoted. Object detection based on deep learning can automatically extract target features without wasting a lot of time on manual analysis, design and extraction, which greatly improves the detection accuracy and scene applicability. Therefore, vehicle and pedestrian detection technology based on deep learning has set off a research boom.

[0003] The problem that the vehicle and pedestrian detection algorithm needs to solve is: find all vehicles and pedestrians in the image or video frame, including their position and size, generally represented by a rectangular frame. Most of the existing vehicle and pedestrian target detection methods have innovated the target detection network, feature extraction methods, etc. These methods focus on the target behavior itself in the picture and video, but the target behavior information in the picture and video is limited, and the scene information in the picture and video and the correlation between the scene and the target behavior are often ignored. However, the scene information in the picture and video and the correlation between the scene and the target behavior have a direct impact on the accuracy of the vehicle and pedestrian detection algorithm. Summary of the invention

[0004] In order to overcome the shortcomings of the existing technology, the present invention proposes a complex multi-target hierarchical and graded joint precise detection method for smart transportation, which can achieve precise detection of various targets around the clock and in all weather conditions while ensuring the cost.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] The present invention proposes a complex multi-target hierarchical and joint precise detection method for smart transportation. The smart transportation environment is a three-dimensional scene space including time, weather, and target scenes. The specific steps of the method are as follows:

[0007] Step 1: Train YOLOv3 for three single-dimensional scenarios: time, weather, and target, and obtain three types of target detection models for single-dimensional scenarios: single-dimensional time detection model, single-dimensional weather detection model, and single-dimensional target detection model.

[0008] Step 2, based on the one-dimensional time detection model, the one-dimensional weather detection model and the one-dimensional target detection model, target detection is performed on the image to be detected respectively;

[0009] Step 3: perform hierarchical combination on the target detection results of step 2 and output the final detection result of the image to be detected.

[0010] Furthermore, in step 1, three types of target detection models for single-dimensional scenes are constructed, as follows:

[0011] (1) According to the needs, the time-related single-dimensional scenes are divided into m1 category, the weather-related single-dimensional scenes are divided into m2 category, and the target-related single-dimensional scenes are divided into m3+1 category;

[0012] (2) For each category of the time-related one-dimensional scene, the training samples of each category are labeled, and YOLOv3 is trained to obtain m1 one-dimensional time detection models; for each category of the weather-related one-dimensional scene, the training samples of each category are labeled, and YOLOv3 is trained to obtain m2 one-dimensional weather detection models; for each category of the target-related one-dimensional scene, the training samples of each category are labeled, and YOLOv3 is trained to obtain m3 one-dimensional target detection models; among them, the one-dimensional time detection model and the one-dimensional weather detection model both include m3+1 outputs corresponding to m3 types of targets and 1 other type, and the one-dimensional target detection model includes 2 outputs corresponding to 1 type of target and 1 other type.

[0013] Furthermore, in step 2, according to the category of the image to be detected in the time-type single-dimensional scene and the weather-type single-dimensional scene, the corresponding single-dimensional time detection model and single-dimensional weather detection model are selected to perform target detection on the image to be detected.

[0014] Furthermore, the hierarchical union in step 3 is specifically as follows:

[0015] (1) The output results of the m3 single-dimensional target detection models are fused with the output results of the single-dimensional time detection model respectively, and the output results of the m3 single-dimensional target detection models are fused with the output results of the single-dimensional weather detection model at the same time. This is the first-layer combination.

[0016] (2) The two fusion results of the first-layer combination are further combined, which is the second-layer combination.

[0017] Furthermore, when the first layer is combined, the output results of the m3 single-dimensional target detection models and the single-dimensional time detection model / single-dimensional weather detection model are superimposed, and the superimposed results are processed as follows to output the fusion results:

[0018] (i) The overlapping prediction frames are merged into one prediction frame, and other non-overlapping prediction frames are kept unchanged; the principle of the merging is as follows:

[0019] (1) If, among the overlapping prediction frames, the prediction frame corresponding to the one-dimensional time detection model / one-dimensional weather detection model and any prediction frame corresponding to the one-dimensional target detection model belong to the same category and their category is not other, then the category of the merged prediction frame is the same category;

[0020] (2) If, among the overlapping prediction frames, the prediction frames corresponding to the one-dimensional time detection model / one-dimensional weather detection model and all the prediction frames corresponding to the one-dimensional target detection model belong to the category of others, then the category of the merged prediction frame is others;

[0021] (3) If the categories of the overlapping prediction frames corresponding to the one-dimensional time detection model / one-dimensional weather detection model and the prediction frames corresponding to any one-dimensional target detection model are inconsistent, then P uothe The category corresponding to the maximum value among the three is taken as the category to which the merged prediction box belongs, and the probability of the category to which the merged prediction box belongs is the maximum value; among them, Indicates that the prediction box corresponding to the one-dimensional time detection model / one-dimensional weather detection model belongs to category j l The probability of l Represents one of the m3+1 outputs of the one-dimensional time detection model / one-dimensional weather detection model; Indicates that the prediction box of the corresponding single-dimensional target detection model belongs to category j k The joint probability of j k represents one of the m3 targets, P uother Indicates the joint probability that the predicted box of the corresponding one-dimensional target detection model belongs to other categories, Indicates that the output includes j k The output of the one-dimensional target detection model is j k The joint probability of Indicates that the output includes j k The output of the one-dimensional target detection model is the probability of other, k = 1, 2, ..., m3; if the output includes j k The single-dimensional target detection model does not output the prediction box, then

[0022] (4) If there is no prediction frame corresponding to the single-dimensional time detection model / single-dimensional weather detection model in the overlapping prediction frames, then: (a) if all prediction frames belong to the same category, the merged prediction frame belongs to the category of others; (b) if there is a prediction frame B that belongs to a category different from other prediction frames, the merged prediction frame belongs to the category of the prediction frame B;

[0023] (ii) For a single prediction box that does not overlap with it, the prediction box is temporarily saved and the probability of its category remains unchanged.

[0024] Furthermore, the second layer combination is specifically as follows: superimposing the two fusion results of the first layer combination, and performing the following processing on the superimposed result to output the final detection result:

[0025] (I) For overlapping prediction boxes, merge them into one prediction box, and determine the category of the merged prediction box according to the following rules:

[0026] (1) If the overlapping prediction boxes belong to the same category, the category of the merged prediction box is the same category;

[0027] (2) If the overlapping prediction boxes belong to different categories, compare the corresponding degrees of conformity of the prediction boxes:

[0028] (a) If the degrees of conformity are different, the category and probability of the merged prediction box are the category and probability of the prediction box with the smaller degree of conformity;

[0029] (b) If the degree of conformity is the same, the category and probability of the merged prediction box are the category and probability of the prediction box with the larger probability;

[0030] (ii) For a single prediction box that does not overlap with it, if the probability of the category to which the prediction box belongs is less than the false detection threshold, the prediction box is deleted; otherwise, the prediction box is retained and the probability remains unchanged.

[0031] Furthermore, when the first layer / second layer are combined, if the overlapping prediction frames have the same category, the probability of the category to which the merged prediction frame belongs is updated, and the updated probability is Among them, q represents the number of prediction boxes of the same category in the overlapping prediction boxes, represents the probability of the category of the o-th prediction box with the same category, and Δ represents the compensation value.

[0032] Further, the formula of the false detection threshold d1 is defined as follows:

[0033] d1=d+δ×BV

[0034] Among them, d is the false detection basic threshold, δ is the coefficient, BV is the background difference, H Currenti , S Currenti 、V Currenti They represent the number of i values ​​of the three components of the image to be detected, H, Saturation S, and Lightness V. Basei , S Basei 、V Basei It represents the number of i values ​​of the three components H, S, and V of the reference image, and H+S+V represents the total number of H, S, and V components in the image to be detected and the reference image.

[0035] Furthermore, the compliance is divided into two categories: time-related single-dimensional scenario compliance and weather-related single-dimensional scenario compliance:

[0036] APM of the temporal one-dimensional scene conformity of the image to be detected TCurrent =|ADER T -DER Current |

[0037] APM of the weather-related single-dimensional scene conformity of the image to be detected WCurrent =|ADER W -DER Current |

[0038] Among them, ADER T is the average dynamic change rate of the time-based single-dimensional scene, ADER W is the average dynamic change rate of the weather-related single-dimensional scene, DER Current is the dynamic change rate of the image to be detected;

[0039] If APM TCurrent >APM WCurrent , the category to which the merged prediction box belongs is the category of the output result of the corresponding one-dimensional weather detection model, otherwise the category to which the merged prediction box belongs is the category of the output result of the corresponding one-dimensional time detection model.

[0040] Furthermore, the dynamic change rate is the average of the grayscale distribution change rates between the image to be detected and its two preceding and following frames. The grayscale distribution change rate between the image to be detected and the preceding / following frame images is:

[0041]

[0042] Among them, PR grayrC It indicates the proportion of pixels with gray value r in the image to be detected among all pixels in the image to be detected. Indicates the proportion of pixels with gray value r in the front / back frame image of the image to be detected to all pixels in the front / back frame image;

[0043] The average dynamic change rate of the time / weather single-dimensional scene is the average value of the dynamic change rates between a plurality of groups of consecutive 3-frame images.

[0044] Beneficial effect: The complex multi-target hierarchical and graded joint precise detection method for intelligent transportation proposed by the present invention can realize accurate detection of various types of targets all day and all weather while ensuring the cost. First, three types of target detection models for single-dimensional scenes are constructed, and then the scenes of the two dimensions of time and weather are used as prior knowledge. According to the current time and weather, the corresponding single-dimensional time detection model and single-dimensional weather detection model are selected, and then these two models are combined with m3 single-dimensional target detection models for hierarchical joint detection. Finally, a hierarchical joint detection mechanism is proposed as the standard for hierarchical joint detection. For the first layer of joint detection, the target category is determined in turn according to the two different levels of dimension and probability (the priority decreases in sequence); for the second layer of joint detection, the target category is determined in turn according to the two different levels of compliance and probability (the priority decreases in sequence). The present invention can be widely used in the field of machine vision for hierarchical and graded joint precise detection of complex multi-targets in transportation, and can realize accurate detection of various types of targets while ensuring the cost, and has a very broad application prospect. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Schematic diagram of the process of building three types of detection models for single-dimensional scenarios;

[0046] Figure 2 It is a flowchart of hierarchical union;

[0047] Figure 3 This is a flowchart of the first-layer joint process;

[0048] Figure 4 This is a flow chart of the second-layer union;

[0049] Figure 5 This is a flow chart of the complex multi-target hierarchical and graded joint precise detection method for smart transportation. DETAILED DESCRIPTION

[0050] In order to describe in more detail the complex multi-target hierarchical and graded joint precise detection method for smart transportation proposed by the present invention, it is further elaborated below with reference to the accompanying drawings and specific embodiments.

[0051] In one embodiment, Figure 5 As shown, a complex multi-target hierarchical and graded joint precise detection method for smart transportation is provided. In this method, the smart transportation environment is regarded as a three-dimensional scene space including time, weather, and target scenes. The specific steps are as follows:

[0052] Step 1: Train YOLOv3 for three single-dimensional scenarios: time, weather, and target, and obtain three types of target detection models for single-dimensional scenarios: single-dimensional time detection model, single-dimensional weather detection model, and single-dimensional target detection model.

[0053] Step 2, based on the one-dimensional time detection model, the one-dimensional weather detection model and the one-dimensional target detection model, target detection is performed on the image to be detected respectively;

[0054] Step 3: perform hierarchical combination on the target detection results of step 2 and output the final detection result of the image to be detected.

[0055] In one embodiment, Figure 1 As shown in Figure 1, three types of target detection models for single-dimensional scenes are constructed in step 1, as follows:

[0056] (1) According to the needs, the time-related single-dimensional scenes are divided into m1 category, the weather-related single-dimensional scenes are divided into m2 category, and the target-related single-dimensional scenes are divided into m3+1 category;

[0057] (2) For each category of the time-related one-dimensional scene, the training samples of each category are labeled, and YOLOv3 is trained to obtain m1 one-dimensional time detection models; for each category of the weather-related one-dimensional scene, the training samples of each category are labeled, and YOLOv3 is trained to obtain m2 one-dimensional weather detection models; for each category of the target-related one-dimensional scene, the training samples of each category are labeled, and YOLOv3 is trained to obtain m3 one-dimensional target detection models; among them, the one-dimensional time detection model and the one-dimensional weather detection model both include m3+1 outputs corresponding to m3 types of targets and 1 other type, and the one-dimensional target detection model includes 2 outputs corresponding to 1 type of target and 1 other type.

[0058] In one embodiment, step 1 includes the following steps:

[0059] Step S101, constructing a three-dimensional scene space;

[0060] Aiming at the complex traffic environment: different time scenes such as day / night / early morning / evening, etc., different weather scenes such as sunny / cloudy / rainy / snowy, etc., and multiple traffic targets: motor vehicles, non-motor vehicles, pedestrians and other target types, an idea method of constructing a three-dimensional scene space is proposed. The scenes of these three dimensions are: time, weather, and target-type single-dimensional scenes. Among them, the time-type single-dimensional scenes can be divided into early morning / evening / …, with a total of m1 categories; the weather-type single-dimensional scenes can be divided into sunny / cloudy / …, with a total of m2 categories; the target-type single-dimensional scenes can be divided into motor vehicles / non-motor vehicles / pedestrians / … / others, with a total of m3+1 categories.

[0061] Step S102, selecting sample data;

[0062] For single-dimensional scenarios such as time, weather, and target, select corresponding sample data respectively.

[0063] Step S103, sample labeling;

[0064] The data labeling tool labelimg is used to label the sample data collected by S102.

[0065] Step S104, model training;

[0066] The labeled and organized dataset is used for YOLOv3 model training, and a total of m1 single-dimensional time detection models, m2 single-dimensional weather detection models, and m3 single-dimensional target detection models can be obtained. The outputs of the single-dimensional time detection model / single-dimensional weather detection model are all motor vehicles / non-motor vehicles / pedestrians / … / others, with a total of m3+1 outputs; the output of the single-dimensional target detection model is one of "motor vehicles / others", "non-motor vehicles / others", "pedestrians / others"….

[0067] In one embodiment, in step 2, according to the category of the image to be detected in the time-type single-dimensional scene and the weather-type single-dimensional scene, the corresponding single-dimensional time detection model and the single-dimensional weather detection model are selected to perform target detection on the image to be detected.

[0068] In one embodiment, Figure 2 As shown, the hierarchical union in step 3 mainly includes the following steps:

[0069] Step S201, first level joint

[0070] The output results of the m3 single-dimensional target detection models are fused with the output results of the single-dimensional time detection model. At the same time, the output results of the m3 single-dimensional target detection models are fused with the output results of the single-dimensional weather detection model. This is the first layer of joint detection.

[0071] Step S202: Second level association

[0072] The two fusion results of the first-layer joint detection are further fused and output, which is the second-layer joint detection.

[0073] In one embodiment, since the one-dimensional time detection model, the one-dimensional weather detection model, and the one-dimensional target detection model are oriented to the target class one-dimensional scene for sample annotation and training, the one-dimensional target detection model has a higher target recognition accuracy in the image to be detected than the one-dimensional time detection model / one-dimensional weather detection model. The detection target is j k The single-dimensional target detection model recognizes the target as j k There is no false detection or missed detection, and the target is not j k There is no false detection but there may be missed detection (it should be noted here that jk For any of the following categories, 1≤k≤m3), the one-dimensional time detection model / one-dimensional weather detection model may misdetect m3+1 target categories, but will only miss them when the category is other. In the first layer of combination, the output results of the m3 one-dimensional target detection models and the one-dimensional time detection model / one-dimensional weather detection model are superimposed, and the superimposed results are processed as follows: Figure 3 After processing according to the process shown, the fusion result is output:

[0074] (i) The overlapping prediction frames are merged into one prediction frame, and other non-overlapping prediction frames are kept unchanged; the principle of the merging is as follows:

[0075] (1) If, among the overlapping prediction frames, the prediction frame corresponding to the one-dimensional time detection model / one-dimensional weather detection model and any prediction frame corresponding to the one-dimensional target detection model belong to category A, then the category of the merged prediction frame is A. It should be noted here that A is any one of the categories of motor vehicles, non-motor vehicles, pedestrians, etc. but not others.

[0076] (2) If, among the overlapping prediction frames, the prediction frames corresponding to the one-dimensional time detection model / one-dimensional weather detection model and all the prediction frames corresponding to the one-dimensional target detection model belong to the category of others, then the merged prediction frame belongs to the category of others.

[0077] In the above two cases (1) and (2), the probability of the category to which the merged prediction box belongs is updated as follows:

[0078]

[0079] Among them, q represents the number of prediction boxes of the same category in the overlapping prediction boxes, represents the probability of the category of the o-th prediction box with the same category, and Δ represents the compensation value.

[0080] (3) If the categories of the overlapping prediction frames corresponding to the one-dimensional time detection model / one-dimensional weather detection model and the prediction frames corresponding to any one-dimensional target detection model are inconsistent, then P uoth The category corresponding to the maximum value among the three is taken as the category to which the merged prediction box belongs, and the probability of the category to which the merged prediction box belongs is the maximum value.

[0081] In general, the detection target is j k A single-dimensional target detection model with the output target category j k The probability of The probability of outputting the target category as other is expressed as Where other indicates that the output target category is other, 1≤k≤m3. In particular, when detecting a certain target, no prediction box is output, that is, missed detection.

[0082] In general, the output target category of the one-dimensional time detection model is j l The probability of Where T represents time, j l Refers to one of the m3+1 target categories. In particular, when detecting a target, no prediction box is output, that is, missed detection. In this case, j l For other, the probability is

[0083] In general, the output target category of the one-dimensional weather detection model is j l′ The probability of Where W represents time, j l′ Refers to one of the m3+1 target categories. It should be noted here that j l and j l′ They may be the same or different. In particular, when detecting a certain target, no prediction box is output, that is, missed detection. In this case, j l′ For other, the probability is

[0084] Therefore, the output category of the one-dimensional target model is j k The joint probability of is calculated as follows:

[0085]

[0086] The joint probability of the output category of the single-dimensional target model being other is calculated as follows:

[0087]

[0088] (4) If there is no prediction frame corresponding to the single-dimensional time detection model / single-dimensional weather detection model in the overlapping prediction frames, then: (a) if all prediction frames belong to the same category, the merged prediction frame belongs to the category of others; (b) if there is a prediction frame A that belongs to a category different from other prediction frames, the merged prediction frame belongs to the category of the prediction frame A;

[0089] (ii) For a single prediction box that does not overlap with it, the prediction box is temporarily saved and the probability of its category remains unchanged.

[0090] In one embodiment, the two fusion results of the first layer are superimposed in the second layer combination, and the superimposed result is processed as follows: Figure 4 After processing according to the process shown, the final detection result is output:

[0091] (I) For overlapping prediction boxes, merge them into one prediction box, and determine the category of the merged prediction box according to the following rules:

[0092] (1) If the overlapping prediction boxes belong to the same category, the category of the merged prediction box is the same category; in this case, the probability of the category to which the merged prediction box belongs is updated as:

[0093]

[0094] Among them, q represents the number of prediction boxes of the same category in the overlapping prediction boxes, represents the probability of the category of the o-th prediction box with the same category, and Δ represents the compensation value.

[0095] (2) If the overlapping prediction boxes belong to different categories, compare the corresponding degrees of conformity of the prediction boxes:

[0096] (a) If the degrees of conformity are different, the category and probability of the merged prediction box are the category and probability of the prediction box with the smaller degree of conformity;

[0097] (b) If the degree of conformity is the same, the category and probability of the merged prediction box are the category and probability of the prediction box with the larger probability;

[0098] (ii) For a single prediction box that does not overlap with it, if the probability of the category to which the prediction box belongs is less than the false detection threshold, the prediction box is deleted; otherwise, the prediction box is retained and the probability remains unchanged.

[0099] In one embodiment, the formula of the false detection threshold d1 is defined as follows:

[0100] d1=D(BV)=d+δ×BV

[0101] Where d is the false detection threshold, δ is the coefficient, BV is the background difference, H Currenti , S Currenti 、V Currenti They represent the number of i values ​​of the three components of the image to be detected, H, Saturation S, and Lightness V. Basei , S Basei 、V Basei It represents the number of i values ​​of the three components H, S, and V of the reference image; H+S+V represents the total number of H, S, and V components in the image to be detected and the reference image, which is used for normalization.

[0102] In one embodiment, the conformity is obtained by the dynamic change rate, which is defined as the average of the grayscale distribution change rates between the image to be detected and its two preceding and following frames, where the grayscale distribution change rate between the image to be detected and the preceding / following frame images is:

[0103]

[0104] PR grayrC It indicates the proportion of pixels with gray value r in the image to be detected among all pixels in the image to be detected. It indicates the proportion of the pixels with gray value r in the previous / next frame image of the image to be detected to all the pixels in the previous / next frame image. It can be seen that the dynamic change rate is the gray distribution change rate between three consecutive frames of images.

[0105] According to the average dynamic change rate ADER of the corresponding time scene of the image to be detected T , the average dynamic change rate ADER of the corresponding weather scene W And the dynamic change rate DER of the image to be detected Current , calculate the compliance:

[0106] APM TCurrent =|ADER T -DER Current |

[0107] APM WCurrent =|ADER W -DER Current |

[0108] APM WCurrent APM is the weather-related single-dimensional scene conformity of the image to be detected. TCurrent ADER is the temporal one-dimensional scene conformity of the image to be detected. T is the average dynamic change rate of the time-based single-dimensional scene, ADER W is the average dynamic change rate of the weather-related single-dimensional scene, DER Current The average dynamic change rate of the time / weather single-dimensional scene is the average of the dynamic change rates between a number of groups of three consecutive frames of images.

[0109] Since the current scene is a scene where time and weather are intersected, the dynamic change rate of the current scene is subtracted from the average dynamic change rate of the corresponding time scene and the corresponding weather scene, and then the absolute value is taken to get the conformity. Then, it is judged whether the image to be detected is closer to the corresponding time scene or the corresponding weather scene. The similar scene can more accurately describe the image to be detected, so the model established by the similar scene can more accurately predict the target. Therefore, if APM Tnow>APM Wnow , the weather scene output shall prevail, otherwise the time scene output shall prevail.

[0110] The above description is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any person familiar with the technology can understand and think of any changes or substitutions within the technical scope disclosed by the present invention, which should be included in the scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A complex multi-target hierarchical joint precision detection method for smart transportation, characterized by: The smart traffic environment is a three-dimensional scene space including time, weather, and target scenes. The specific steps of the method are as follows: Step 1: Train YOLOv3 for three single-dimensional scenarios: time, weather, and target, and obtain three types of target detection models for single-dimensional scenarios: single-dimensional time detection model, single-dimensional weather detection model, and single-dimensional target detection model. Step 2, based on the one-dimensional time detection model, the one-dimensional weather detection model and the one-dimensional target detection model, target detection is performed on the image to be detected respectively; Step 3, performing hierarchical combination on the target detection results of step 2, and outputting the final detection result of the image to be detected; The hierarchical union in step 3 is specifically as follows: (1) The output results of the m3 single-dimensional target detection models are fused with the output results of the single-dimensional time detection model respectively, and the output results of the m3 single-dimensional target detection models are fused with the output results of the single-dimensional weather detection model at the same time. This is the first-layer combination. (2) The two fusion results of the first-layer combination are further combined, which is the second-layer combination; When the first layer is combined, the output results of the m3 single-dimensional target detection models and the single-dimensional time detection model / single-dimensional weather detection model are superimposed, and the superimposed results are processed as follows to output the fusion results: (i) The overlapping prediction frames are merged into one prediction frame, and other non-overlapping prediction frames are kept unchanged; the principle of the merging is as follows: (1) If, among the overlapping prediction frames, the prediction frame corresponding to the one-dimensional time detection model / one-dimensional weather detection model and any prediction frame corresponding to the one-dimensional target detection model belong to the same category and their category is not other, then the category of the merged prediction frame is the same category; (2) If, among the overlapping prediction frames, the prediction frames corresponding to the one-dimensional time detection model / one-dimensional weather detection model and all the prediction frames corresponding to the one-dimensional target detection model belong to the category of others, then the category of the merged prediction frame is others; (3) If the categories of the overlapping prediction frames corresponding to the one-dimensional time detection model / one-dimensional weather detection model and the prediction frames corresponding to any one-dimensional target detection model are inconsistent, then The category corresponding to the maximum value among the three is taken as the category to which the merged prediction box belongs, and the probability of the category to which the merged prediction box belongs is the maximum value; among them, Indicates that the prediction box corresponding to the one-dimensional time detection model / one-dimensional weather detection model belongs to category j l The probability of l Represents one of the m3+1 outputs of the one-dimensional time detection model / one-dimensional weather detection model; Indicates that the prediction box corresponding to the single-dimensional target detection model belongs to category j k The joint probability of j k represents one of the m3 targets, P uothe Indicates the joint probability that the predicted box of the corresponding one-dimensional target detection model belongs to other categories, Indicates that the output includes j k The output of the one-dimensional target detection model is j k The joint probability of Indicates that the output includes j k The output of the one-dimensional target detection model is the probability of other, k = 1, 2, ..., m3; if the output includes j k The one-dimensional target detection model does not output the prediction box, then (4) If there is no prediction frame corresponding to the single-dimensional time detection model / single-dimensional weather detection model in the overlapping prediction frames, then: (a) if all prediction frames belong to the same category, the merged prediction frame belongs to the category of others; (b) if there is a prediction frame B that belongs to a category different from other prediction frames, the merged prediction frame belongs to the category of the prediction frame B; (ii) For a single prediction box that does not overlap with it, the prediction box is temporarily saved and the probability of its category remains unchanged; The second layer combination is as follows: superimpose the two fusion results of the first layer combination, and perform the following processing on the superimposed result to output the final detection result: (I) For overlapping prediction boxes, merge them into one prediction box, and determine the category of the merged prediction box according to the following rules: (1) If the overlapping prediction boxes belong to the same category, the category of the merged prediction box is the same category; (2) If the overlapping prediction boxes belong to different categories, compare the corresponding degrees of conformity of the prediction boxes: (a) If the degrees of conformity are different, the category and probability of the merged prediction box are the category and probability of the prediction box with the smaller degree of conformity; (b) If the degree of conformity is the same, the category and probability of the merged prediction box are the category and probability of the prediction box with the larger probability; (ii) For a single prediction box that does not overlap with it, if the probability of the category to which the prediction box belongs is less than the false detection threshold, the prediction box is deleted; otherwise, the prediction box is retained and the probability remains unchanged.

2. The complex multi-target hierarchical and graded joint precise detection method for smart transportation according to claim 1 is characterized in that: In step 1, three types of target detection models for single-dimensional scenes are constructed, as follows: (1) According to the needs, the time-related single-dimensional scenes are divided into m1 category, the weather-related single-dimensional scenes are divided into m2 category, and the target-related single-dimensional scenes are divided into m3+1 category; (2) For each category of the time-related one-dimensional scene, the training samples of each category are labeled, and YOLOv3 is trained to obtain m1 one-dimensional time detection models; for each category of the weather-related one-dimensional scene, the training samples of each category are labeled, and YOLOv3 is trained to obtain m2 one-dimensional weather detection models; for each category of the target-related one-dimensional scene, the training samples of each category are labeled, and YOLOv3 is trained to obtain m3 one-dimensional target detection models; among them, the one-dimensional time detection model and the one-dimensional weather detection model both include m3+1 outputs corresponding to m3 types of targets and 1 other type, and the one-dimensional target detection model includes 2 outputs corresponding to 1 type of target and 1 other type.

3. The complex multi-target hierarchical and graded joint precise detection method for smart transportation according to claim 2 is characterized in that: In the step 2, according to the category of the image to be detected in the time-type single-dimensional scene and the weather-type single-dimensional scene, the corresponding single-dimensional time detection model and the single-dimensional weather detection model are selected to perform target detection on the image to be detected.

4. The complex multi-target hierarchical and combined precise detection method for intelligent transportation according to claim 1 is characterized in that: When the first layer / second layer are combined, if the overlapping prediction frames have the same category, the probability of the category to which the merged prediction frame belongs is updated, and the updated probability is Among them, q represents the number of prediction boxes of the same category in the overlapping prediction boxes, represents the probability of the category of the o-th prediction box with the same category, and Δ represents the compensation value.

5. The complex multi-target hierarchical and graded joint precise detection method for smart transportation according to claim 4 is characterized in that: The formula of the false detection threshold d1 is defined as follows: d1=d+δ×BV Among them, d is the false detection basic threshold, δ is the coefficient, BV is the background difference, H Currenti , S Currenti 、V Currenti Respectively represent the number of i values ​​of the three components of the image to be detected, H, Saturation S, and Lightness V. Basei , S Basei 、V Basei It represents the number of i values ​​of the three components H, S, and V of the reference image, and H+S+V represents the total number of H, S, and V components in the image to be detected and the reference image.

6. The complex multi-target hierarchical and graded joint precise detection method for smart transportation according to claim 4 is characterized in that: The compliance is divided into two categories: time-related single-dimensional scenario compliance and weather-related single-dimensional scenario compliance: APM of the temporal one-dimensional scene conformity of the image to be detected TCurrent =|ADER T -DER Current | APM of the weather-related single-dimensional scene conformity of the image to be detected WCurrent =|ADER W -DER Current | Among them, ADER T is the average dynamic change rate of the time-based single-dimensional scene, ADER W is the average dynamic change rate of the weather-related single-dimensional scene, DER Current is the dynamic change rate of the image to be detected; If APM TCurrent >APM WCurrent , the category to which the merged prediction box belongs is the category of the output result of the corresponding one-dimensional weather detection model, otherwise the category to which the merged prediction box belongs is the category of the output result of the corresponding one-dimensional time detection model.

7. The complex multi-target hierarchical and combined precise detection method for intelligent transportation according to claim 6 is characterized in that: The dynamic change rate is the average of the grayscale distribution change rates between the image to be detected and the two frames before and after it. The grayscale distribution change rate between the image to be detected and the previous / next frame image is: Among them, PR grayrC It indicates the proportion of pixels with gray value r in the image to be detected among all pixels in the image to be detected. Indicates the proportion of pixels with gray value r in the front / back frame image of the image to be detected to all pixels in the front / back frame image; The average dynamic change rate of the time / weather single-dimensional scene is the average value of the dynamic change rates between a plurality of groups of consecutive 3-frame images.

Images