Visibility detection method and device based on ordinal number consistency constraint

Abstract

The invention relates to the technical field of visibility detection, in particular to a visibility detection method and device based on ordinal number consistency constraint. The method comprises the step of constraining the consistency of classifier results by converting a traditional visibility grade classification task into a plurality of ordered binary classifiers. Meanwhile, a self-adaptive ordinal number consistency constraint is further provided, and the training samples are weighted by using the consistency, so that the weight for detecting inconsistent samples is increased, and the visibility classification accuracy is improved. According to the visibility detection method based on ordinal number consistency constraint, training and testing are carried out on image data collected by video monitoring equipment on an expressway in a centralized mode, the effect of the method is checked through an ablation experiment, and an experiment result shows that the visibility detection method based on ordinal number consistency constraint can effectively improve the classification accuracy of a visibility detection model.

Description

Technical Field

[0001] The present invention relates to the technical field of visibility detection, in particular to a visibility detection method and device based on ordinal consistency constraint. Background Art

[0002] Visibility, as a key indicator for measuring atmospheric transparency, plays a crucial role in fields such as environmental monitoring, traffic management, and public safety. Traditional visibility measurement methods, such as transmissometers, although having high precision, have limitations such as high cost and difficulty in site layout, making it difficult to achieve large-scale and high-density monitoring. With the popularization of video surveillance technology and the rapid development of image processing algorithms, visibility inversion technology based on video images has become a new research hotspot. There have been many studies on video-based visibility inversion technology in China. For example, a study [Liu Dongwei, Mu Haizhen, He Qianshan, et al. A low visibility recognition algorithm based on surveillance video [J]. Journal of Applied Meteorological Science, 2022, 33(4): 501-512. LIU D W, MU H Z, HE Q S, et al. A low visibility recognition algorithm based on surveillance video [J]. Journal of Applied Meteorological Science, 2022, 33(4): 501-512.] proposed a low visibility recognition algorithm based on real-scene image conversion and convolutional neural network. By analyzing the gradient, saturation, and brightness information of images, effective classification of visibility levels was achieved. In addition, satellite remote sensing technology has also been used to invert surface visibility. By combining MODIS satellite aerosol optical depth and the aerosol elevation field simulated by GEOS-Chem, a visibility inversion algorithm [Zhang Yan, Li Jing. Retrieval of surface visibility using satellite-based aerosol measurements [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2020, 56(02): 231-241. Zhang Y, Li J. Retrieval of surface visibility using satellite-based aerosol measurements [J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2020, 56(2): 231-241.] was proposed.Some studies have proposed traffic visibility estimation models based on different scenarios [Liu Qiyang, Qiao Fengxue, Chen Bo, et al., 2022. Application of traffic visibility estimation models in different scenarios [J]. Transactions of Atmospheric Sciences, 45(2): 179-190.]. These models comprehensively utilize meteorological observation data and image features, effectively reducing the dependence on observation instruments and improving the flexibility and efficiency of visibility monitoring.

[0003] In recent years, with the rapid development of deep learning technology, visibility detection models based on deep learning models have achieved good performance and received extensive attention from researchers. Some researchers regard visibility estimation as a regression task. Li et al. [Li Q, Xie B. Visibility estimation using a single image[C] / / ComputerVision: Second CCF Chinese Conference, CCCV 2017, Tianjin, China, October 11–14, 2017, Proceedings, Part I. Springer Singapore, 2017: 343-355.] assumed that the extinction coefficient of a fog-free image is approximately constant. Combining the dark channel prior [He K, Sun J, Tang X. Single image hazeremoval using dark channel prior[J]. IEEE transactions on pattern analysisand machine intelligence, 2010, 33(12): 2341-2353], the ratio of the two extinction coefficients in the foggy image and the fog-free image can be calculated. Based on this, after calculating the extinction coefficient of the foggy image, the Koschmieder law [Koschmieder H. Theorie der horizontalen Sichtweite[M]. Keim & Nemnich,1925] and the contrast attenuation law are combined to calculate the visibility on real urban scene images. Subsequently, the visibility detection method was further improved by using transmission rate optimization based on edge degradation [Li Q, Li Y, Xie B. Single image-based scene visibility estimation[J]. Ieee Access, 2019, 7: 24430-24439]. This method assumes that the extinction coefficient of a fog-free image is approximately constant, and there is a large error in calculating the visibility based on the theory of atmospheric physics, and the estimation of visibility is not accurate enough.You [You Y, Lu C, Wang W, et al. Relative CNN-RNN: Learning relative atmospheric visibility from images[J]. IEEE Transactions on Image Processing, 2018, 28(1): 45-55] and others directly estimated the visibility value of the image. This method constructs a CNN-RNN model, combines the support vector machine (SVM) to learn the relative visibility of the image pair, and thus estimates the absolute visibility. This method first considered the ordinal relationship between the visibilities of the image pair, but a large amount of data preparation work was required in the early stage, and end-to-end training could not be achieved. Lo [Lo W L, Zhu M, Fu H. Meteorology visibility estimation by using multi-support vector regression method[J]. Journal of Advances in Information Technology Vol, 2020, 11(2): 40-47] and others proposed a sub-region visibility estimation method. This method selects effective sub-regions from the images of landmark objects with different visibility ranges, uses the support vector machine (SVM) for feature classification, and uses the multi-head support vector regression (MSVR) representing different visibility ranges to directly estimate the visibility. Subsequently, Lo et al. [Li J, Lo W L, Fu H, et al. A transfer learning method for meteorological visibility estimation based on feature fusion method[J]. Applied Sciences, 2021, 11(3): 997.] improved this method: used the gray-scale weighted average operation to determine multiple effective sub-regions of the image to estimate the visibility of each sub-region, and used weight fusion to estimate the visibility of the entire image, further reducing the visibility estimation error.Xun et al. [Xun L, Zhang H, Yan Q, et al. VISOR-NET: visibility estimation based on deep ordinal relative learning under discrete-level labels[J]. Sensors, 2022, 22(16): 6227] proposed a method for visibility level classification and visibility estimation using the ordinal information and relative relationships of images. This method only achieved visibility level classification on the collected highway images. Yao [Yao S, Huang B. Extraction of aerosol optical extinction properties from a smartphone photograph to measure visibility[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60: 1-13] et al. directly estimated visibility by extracting AOEPs (i.e., local medium transmission rate and local medium extinction coefficient) from images and combining atmospheric physics theory. This method requires detecting the vanishing point of the image to select appropriate AOEPs for visibility regression. Due to the influence of light, occlusions, camera perspective, etc., it is difficult to detect the vanishing point. You [You J, Jia S, Pei X, et al. DMRVisNet: Deep multihead regression network for pixel-wise visibility estimation under foggy weather[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(11): 22354-22366] et al. used an encoder and three decoders to extract air light, transmittance, and depth information, and calculated the pixel-level visibility map and image-level visibility on synthetic images in combination with the Koschmieder's law. This method can be applied to real road scenes for visibility estimation.In the same year, Zhang [Zhang F, Yu T, Li Z, et al. Deep quantified visibility estimation for traffic image[J]. Atmosphere, 2022, 14(1): 61.] et al. proposed an end-to-end deep quantified visibility estimation network composed of a transmittance estimation module, a depth estimation module, and an extinction co-estimation module. This method realized end-to-end direct visibility estimation for highway images based on atmospheric physics methods.

[0004] In addition to regarding visibility estimation as a regression task, some researchers regard visibility estimation as a classification task to learn the relationship between image features and visibility levels. In 2017, Li et al. [Li S, Fu H, Lo W L. Meteorological visibility evaluation on webcam weather image using deeplearning features[J]. Int. J. Comput. Theory Eng, 2017, 9(6): 455-461] first used CNN to estimate the visibility of road images captured by cameras. Based on the collected large-scale fog image dataset and visibility annotations, AlexNet [Krizhevsky A, Sutskever I, Hinton G E. ImageNetclassification with deep convolutional neural networks[J]. Communications ofthe ACM, 2017, 60(6): 84-90.] was used to extract visibility features in the images and perform visibility level classification, showing good application results. Vaibhav [Vaibhav V, Konda K R, Kondapalli C, et al. Real-timefog visibility range estimation for autonomous driving applications[C] / / 2020IEEE 23rd International Conference on Intelligent Transportation Systems(ITSC). IEEE, 2020: 1-6.] et al. combined image preprocessing techniques to extract block discrete cosine transform (DCT) features and block-level Shannon entropy features, and input them together with the original image features into the CNN module to achieve visibility level classification.Different from traditional deep learning methods that only use convolutional neural networks (CNNs) to process input images, Liu [Liu J, Chang X, Li Y, et al. STCN-Net: A novel multi-feature stream fusion visibility estimation approach[J]. IEEE Access, 2022, 10: 120329-120342] et al. combined CNNs and Transformers [Liu Z, Lin Y, Cao Y, et al. Swin transformer: Hierarchical vision transformer using shifted windows[C] / / Proceedings of the IEEE / CVF International conference on computer vision. 2021: 10012-10022] to process input images and proposed an STCN-Net for estimating visibility levels. This method designed a new 3D multi-feature stream matrix DDT for visibility estimation, which consists of a transmittance matrix, a dark channel matrix, and a depth matrix. Subsequently, Liu [Liu J, Zhong J, Li Y, et al. Fgs-net: A visibility estimation method based on statistical feature stream in fog area[J]. 2023] et al. improved the model and proposed a new fog area segmentation method to segment the fog area in the input image. Subsequently, the DDT matrix, contrast, average gradient, and brightness features were extracted from the fog area, and the learning features extracted by the Transformer were combined for visibility level classification, achieving better visibility classification performance. However, existing deep learning models have ignored the ordinal relationship between visibility levels. The ordinal regression network [Liu H, Lu J, Feng J, et al. Ordinal deep learning for facial age estimation[J]. IEEE transactions on circuits and systems for video technology, 2017, 29(2): 486-501.] extends the classifier to multiple binary classifiers for ordinal regression, and better captures visibility-related features by leveraging the ordinal relationship between data.However, there is a problem that the detection results of multiple binary classifiers in the ordinal regression network are inconsistent, resulting in unstable detection results of the model. Summary of the Invention

[0005] To solve the technical problem in the prior art that the detection results of multiple binary classifiers in the ordinal regression network are inconsistent, resulting in unstable detection results of the model, the embodiments of the present invention provide a visibility detection method and device based on ordinal consistency constraint. The technical solution is as follows:

[0006] On the one hand, a visibility detection method based on ordinal consistency constraint is provided, characterized in that the method includes:

[0007] S1. Collect image data through a video monitoring device;

[0008] S2. Construct a visibility detection model based on ordinal consistency constraint; the visibility detection model includes two parts: an ordinal regression network based on VGG16 and an ordinal consistency constraint loss function;

[0009] S3. Input the image data into the ordinal regression network based on VGG16, extract the visibility features of the image, and obtain the classification result of the visibility level label;

[0010] S4. Obtain the classification result of the visibility level label, assign different weight information to each sample, calculate the consistency of the intersection over union of the detected visibility label classification result and the true visibility level label through the ordinal consistency constraint loss function, and obtain the accurate visibility level classification result to complete the visibility detection based on ordinal consistency constraint.

[0011] Optionally, S2 includes: the ordinal regression network based on VGG16 includes 13 convolutional network layers, and the ReLU function is used after all convolutional layers; max pooling and batch normalization are used after the first, fourth, seventh, tenth, and last convolutional network layers; learn and extract the mapping of the ordinal relationship between the visibility features and the visibility level from highway images through the ordinal regression based on VGG16;

[0012] Ensure the consistency between the ordinal information learned by the visibility detection model and the true value through the ordinal consistency constraint loss.

[0013] Optionally, in S3, inputting the image data into the ordinal regression network based on VGG16, extracting the visibility features of the image, and obtaining the classification result of the visibility level label includes:

[0014] Input the image data into the ordinal regression network based on VGG16;

[0015] The feature map is reduced to a fixed size through the adaptive average pooling layer of the ordinal regression network based on VGG16;

[0016] Two fully connected layers are used to extract depth feature information related to visibility;

[0017] The visibility level label corresponding to the image data is obtained, and the visibility level label is extended into a vector that retains the order information of the level label;

[0018] The ordinal relationship between the image visibility features and the level labels is learned through ordinal regression to obtain the classification result of the visibility level label.

[0019] Optionally, obtaining the visibility level label corresponding to the image data and extending the visibility level label into a vector that retains the order information of the level label includes:

[0020] Obtain the picture , then the corresponding visibility level label is , where W represents the image width, H represents the image height, C represents the number of image channels, i represents the i-th image, and K represents the number of visibility level categories;

[0021] The visibility level label is encoded into a vector with k elements .

[0022] Optionally, the ordinal relationship between the image visibility features and the level labels is learned through ordinal regression to obtain the classification result of the visibility level label, including:

[0023] The k-th element The calculation method is as shown in the following formula (1):

[0024] (1)

[0025] where ;

[0026] Use classifiers to map the depth visibility feature embedding from two fully connected layers to a binary output visibility label vector consistent with the visibility level label ;

[0027] Calculate the sum of the outputs of the classifiers to obtain the final detected visibility level of the picture as shown in the following formula (2):

[0028] (2).

[0029] Optionally, obtain the visibility level label classification result, assign different weight information to each sample, calculate the consistency of the intersection over union of the detected visibility label classification result and the true visibility level label through the ordinal consistency constraint loss function, and obtain an accurate visibility level classification result to complete the visibility detection based on the ordinal consistency constraint, including:

[0030] Define the ordinal consistency constraint loss function As shown in the following formula (3):

[0031] (3)

[0032] where N represents the number of training set images, represents the exponential open scaling function, represents the ordinal consistency result, represents the cross-entropy loss function.

[0033] Define the exponential square root scaling function As shown in the following formula (4):

[0034] (4)

[0035] For the calculated ordinal consistency result use the exponential square root scaling function as the weight value assigned to the sample;

[0036] Define the cross-entropy function as shown in the following formula (5):

[0037] (5)

[0038] where, represents the number of visibility level categories, represents an indicator function, with the result being 1 when the condition is true and 0 when the condition is false; p represents the model prediction probability; represents the th image; represents the parameters of the model;

[0039] Obtain the visibility label vector and the true visibility label vector ;

[0040] By calculating the intersection over union of the detected visibility label vector and the true visibility level label vector obtain the ordinal consistency result of the visibility level; and use the exponential square root scaling function to assign higher weights to the difficult samples with ordinal inconsistencies, so that the model focuses on the difficult samples;

[0041] Obtain accurate visibility level classification results and complete visibility detection based on ordinal consistency constraints.

[0042] Optionally, by calculating the intersection over union of the detected visibility label vector and the true visibility level label vector to obtain the ordinal consistency result of the visibility level, including:

[0043] Calculate the ordinal consistency of the visibility level according to the following formula (6):

[0044] (6).

[0045] On the other hand, a visibility detection device based on ordinal consistency constraints is provided. This device is applied to the visibility detection method based on ordinal consistency constraints, and the device includes:

[0046] An image data acquisition module for acquiring image data through a video monitoring device;

[0047] A model construction module for constructing a visibility detection model based on ordinal consistency constraints; the visibility detection model includes two parts: an ordinal regression network based on VGG16 and an ordinal consistency constraint loss function;

[0048] A visibility detection module for inputting the image data into the ordinal regression network based on VGG16, extracting the visibility features of the image, and obtaining a visibility level label classification result;

[0049] A detection result calibration module for obtaining the visibility level label classification result, assigning different weight information to each sample, calculating the consistency of the intersection over union of the detected visibility label classification result and the true visibility level label through the ordinal consistency constraint loss function, obtaining accurate visibility level classification results, and completing visibility detection based on ordinal consistency constraints.

[0050] On the other hand, a visibility detection device based on ordinal consistency constraints is provided. The visibility detection device based on ordinal consistency constraints includes: a processor; a memory, and a computer-readable instruction is stored on the memory. When the computer-readable instruction is executed by the processor, any one of the methods in the above-mentioned visibility detection method based on ordinal consistency constraints is implemented.

[0051] On the other hand, a computer-readable storage medium is provided. At least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the methods in the above-mentioned visibility detection method based on ordinal consistency constraints.

[0052] The beneficial effects brought by the technical solutions provided in the embodiments of the present invention at least include:

[0053] In the embodiments of the present invention, a novel visibility detection method based on ordinal consistency constraint is proposed. By transforming the traditional visibility level classification task into multiple ordered binary classifiers, the consistency of the classifier results is constrained. At the same time, an adaptive ordinal consistency constraint is further proposed, and the consistency is used to weight the training samples, increasing the weights of the detected inconsistent samples and improving the accuracy of visibility classification. The designed method has been detailedly experimented on the visibility detection dataset, and the experimental results show that the proposed method can effectively improve the classification accuracy of the visibility detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following described drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0055] Figure 1 It is a schematic flowchart of a visibility detection method based on ordinal consistency constraint provided in the embodiments of the present invention;

[0056] Figure 2 It is a network diagram of visibility level classification based on ordinal consistency provided in the embodiments of the present invention;

[0057] Figure 3 It is an example diagram of all levels of the JS-FHVI and DG-FHVI datasets provided in the embodiments of the present invention;

[0058] Figure 4 Provided in the embodiments of the present invention and The consistency diagram between the parts where the element is 1;

[0059] Figure 5 Provided in the embodiments of the present invention and The consistency diagram between;

[0060] Figure 6 Provided in the embodiments of the present invention and The consistency diagram of the difference between the part where the element is 1 and the part where the element is 0;

[0061] Figure 7 Provided in the embodiments of the present invention and The IoU diagram of;

[0062] Figure 8 The experimental result comparison diagram provided by the embodiment of the present invention;

[0063] Figure 9 The block diagram of the visibility detection device based on ordinal consistency constraint provided by the embodiment of the present invention;

[0064] Figure 10 The structural schematic diagram of the electronic device provided by the embodiment of the present invention. Specific embodiments

[0065] The following combines the accompanying drawings to describe the technical solutions in the present invention.

[0066] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as an "example" in the present invention should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, the use of the word "example" is intended to present concepts in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or either of the two can be selected.

[0067] In the embodiments of the present invention, sometimes subscripts such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meanings they express are the same.

[0068] To make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail in combination with the accompanying drawings and specific embodiments.

[0069] The embodiment of the present invention provides a visibility detection method based on ordinal consistency constraint. This method can be implemented by a visibility detection device based on ordinal consistency constraint. The visibility detection device based on ordinal consistency constraint can be a terminal or a server. As Figure 1 shown in the flowchart of the visibility detection method based on ordinal consistency constraint, as Figure 1 shown, the visibility detection method based on ordinal consistency constraint proposed by the present invention. The processing flow of this method can include the following steps:

[0070] S1. Collect image data through a video monitoring device.

[0071] S2. Build a visibility detection model based on ordinal consistency constraint; the visibility detection model includes two parts: an ordinal regression network based on VGG16 (Visual Geometry Group 16-layer network, a 16-layer deep convolutional neural network of the visual geometry group) and an ordinal consistency constraint loss function.

[0072] In a feasible implementation, there is a natural ordinal relationship between visibility levels. Learning this ordinal relationship based on image visibility features can achieve visibility level classification. Therefore, the present application first constructs a visibility detection model based on ordinal regression. However, for difficult samples where it is difficult to learn the correct ordinal relationship, the model may have a problem that the learned ordinal relationship is inconsistent with the true ordinal information, thus confusing the learning ability of the model, resulting in the inability to effectively learn the correct ordinal information from the extracted visibility features, and further reducing the accuracy of visibility level classification. To solve this problem, the present application further designs a visibility detection model based on ordinal consistency constraint. The overall framework diagram is as Figure 2 shown. This model constrains the ordinal information learned by the model through an adaptive ordinal consistency constraint loss, enabling the model to focus on difficult samples with ordinal inconsistency problems, better learning and processing samples with ordinal inconsistency problems, thereby improving the performance of visibility classification.

[0073] In a feasible implementation, the ordinal regression network based on VGG16 includes 13 convolutional network layers, and the ReLU function is used after all convolutional layers;

[0074] Max pooling and batch normalization are used after the first, fourth, seventh, tenth, and last convolutional network layers to reduce the image dimension;

[0075] Learn and extract the mapping of the ordinal relationship between visibility features and visibility levels from highway images through ordinal regression based on VGG16;

[0076] Ensure the consistency between the ordinal information learned by the visibility detection model and the true value through the ordinal consistency constraint loss.

[0077] S3. Input the image data into the ordinal regression network based on VGG16, extract the visibility features of the image, and obtain the classification result of the visibility level label;

[0078] In a feasible implementation, in S3, inputting the image data into the ordinal regression network based on VGG16, extracting the visibility features of the image, and obtaining the classification result of the visibility level label includes:

[0079] Input the image data into the ordinal regression network based on VGG16;

[0080] Shrink the feature map to a fixed size through the adaptive average pooling layer of the ordinal regression network based on VGG16;

[0081] Extract the depth feature information related to visibility through two fully connected layers;

[0082] Obtain the visibility level label corresponding to the image data, and expand the visibility level label into a vector that retains the order information of the level label; use ordinal regression to learn the ordinal relationship between the image visibility features and the level label, so as to achieve the classification of the visibility level label.

[0083] By using ordinal regression to learn the ordinal relationship between the image visibility features and the level label, obtain the classification result of the visibility level label.

[0084] In a feasible implementation, obtaining the visibility level label corresponding to the image data and expanding the visibility level label into a vector that retains the order information of the level label includes:

[0085] Obtain the picture , then the corresponding visibility level label is , where W represents the image width, H represents the image height, C represents the number of image channels, i represents the i-th image, and K represents the number of visibility level categories;

[0086] Encode the visibility level label into a vector with k elements .

[0087] In a feasible implementation, by using ordinal regression to learn the ordinal relationship between the image visibility features and the level label, obtaining the classification result of the visibility level label includes:

[0088] The k-th element The calculation method is as follows in formula (1):

[0089] (1)

[0090] Among them, ;

[0091] Use classifiers to map the deep visibility feature embedding from two fully connected layers to a binary output visibility label vector consistent with the visibility level label ;

[0092] Calculate the sum of the outputs of the classifiers to obtain the final detected visibility level of the picture as follows in formula (2):

[0093] (2).

[0094] In a feasible implementation, relying only on ordinal regression may cause problems when dealing with difficult samples, especially when the ordinal relationship learned by the model is inconsistent with the true ordinal relationship. For example, the visibility level of an image of a highway , after label expansion , however, the output of the model is , then for the picture the detection visibility level = 3. Although the finally detected visibility level is the same as the true visibility , but this ordinal inconsistency between will confuse the learning process of the model. To solve this problem, this application proposes an ordinal consistency constraint loss function. Through adaptive ordinal consistency constraints, different weight information is assigned to each sample, enabling the model to focus on samples with inconsistent detection ordinal relationships:

[0095] S4. Obtain the visibility level label classification results, assign different weight information to each sample, calculate the consistency of the intersection over union of the detected visibility label classification results and the true visibility level labels through the ordinal consistency constraint loss function, and obtain accurate visibility level classification results to complete visibility detection based on ordinal consistency constraints.

[0096] In a feasible implementation, obtain the visibility level label classification results, calculate the consistency of the intersection over union of the detected visibility label classification results and the true visibility level labels through the ordinal consistency constraint loss function, assign different weight information to each sample through the exponential square root scaling function, obtain accurate visibility level classification results, and complete visibility detection based on ordinal consistency constraints, including:

[0097] Define the ordinal consistency constraint loss function , aiming to minimize the error caused by misclassification for the given input image as shown in the following formula (3):

[0098] (3)

[0099] where N represents the number of training set images, represents the exponential square root scaling function, represents the ordinal consistency result, represents the cross-entropy loss function.

[0100] Define the exponential square root scaling function as shown in the following formula (4):

[0101] (4)

[0102] Use the exponential square root scaling function for the calculated ordinal consistency result as the weight value assigned to the sample;

[0103] Define the cross - entropy function as the following formula (5):

[0104] (5)

[0105] Wherein, represents the number of visibility level categories, represents an indicator function, which results in 1 when the condition is true and 0 when the condition is false; p represents the model prediction probability; represents the th image; represents the parameters of the model;

[0106] Obtain the visibility label vector and the true visibility label vector ;

[0107] By calculating the intersection - over - union of the detected visibility label vector and the true visibility level label vector obtain the ordinal consistency result of the visibility level; and assign higher weights to the difficult samples with ordinal inconsistencies, so that the model focuses on the difficult samples and improves the accuracy of the model's visibility level classification;

[0108] Obtain an accurate visibility level classification result and complete the visibility detection based on ordinal consistency constraints.

[0109] In a feasible implementation, by calculating the intersection - over - union of the detected visibility label vector and the true visibility level label vector obtain the ordinal consistency result of the visibility level, including:

[0110] Calculate the ordinal consistency of the visibility level according to the following formula (6):

[0111] (6).

[0112] In a feasible implementation, the present application details the above - mentioned technical solution through the following actual experimental process:

[0113] First, introduce two data sets collected from real highways: JS - FHVI (visibility images of the 1st foggy highway) and DG - FHVI (visibility images of the 2nd foggy highway). Then, verify and conduct ablation experiments on the proposed method on these two data sets.

[0114] (1) Visibility detection data set:

[0115] JS-FHVI: Collect the image data of 216 video surveillance devices on Highway 1, a total of 7,799 images. These images are randomly divided into a training set, a validation set, and a test set, with ratios of 70%, 10%, and 20% respectively. The detailed information of each level of the images is shown in Table 1 and Figure 3 .

[0116] DG-FHVI: Collect the image data of 133 video surveillance devices on Highway 2, a total of 6,777 highway images. This dataset is closer to the sea, and the climate conditions are different from those of JS-FHVI. In the experiment, these images are randomly divided into a training set, a validation set, and a test set, with ratios of 80%, 10%, and 10% respectively. The detailed information of each level of the images is shown in Table 1 and Figure 3 .

[0117] Table 1 Dataset Information

[0118]

[0119] (2) Evaluation Metrics

[0120] To evaluate the accuracy of visibility detection, the accuracy (Acc), which is commonly used in classification tasks, is used as the evaluation metric in the experiment. It represents the proportion of correctly detected samples among all test samples.

[0121] Four calculation methods for ordinal consistency are defined in the experiment. Formula (6.1) calculates the ordinal consistency of the positions of elements with value 1 in the detected visibility label vector and the true visibility level label vector , focusing on the position information that determines the visibility level label. Formula (6.2) calculates the ordinal consistency of all elements in the detected visibility label vector and the true visibility level label vector , which focuses on the influence of all elements in the vector on the ordinal information. In formula (6.3), the closer the difference between the difference of the positions of elements with value 1 and elements with value 0 in the detected visibility label vector and the true visibility level label vector , the higher the ordinal consistency of the visibility level. Formula (6) calculates the intersection over union of the detected visibility label vector and the true visibility level label vector . The higher the intersection over union, the higher the ordinal consistency of the visibility level.

[0122] (6.1)

[0123] (6.2)

[0124] (6.3)

[0125] (6)

[0126] (3) Visibility detection results

[0127] The detection accuracy of the method of this application on two datasets is shown in Table 2. It can be seen that the performance improvement of the proposed method compared with the original ordinal regression is very obvious.

[0128] Table 2 Detection accuracy of different methods

[0129]

[0130] (4) Ablation experiment

[0131] This application respectively compared the effects of five scaling functions on four ordinal consistency losses, which are linear scaling, exponential scaling, square root of exponential scaling, logarithmic scaling, and softmax scaling. On this basis, two weighting methods were also compared respectively: (Reducing the weight of difficult samples) and (Increasing the weight of difficult samples).

[0132] The results are as follows:

[0133] 4.1. Calculate the consistency between the elements with value 1 in the detected visibility label vector and the true visibility level label vector (different weight scaling functions)

[0134] From the results in Table 3, for the ordinal consistency calculation method of formula (1), exponential scaling and square root of exponential scaling use and to achieve better classification results on both datasets. Linear scaling uses on JS-FHVI to achieve the highest classification accuracy. and The consistency between the elements with value 1 in Figure 4 is as shown in

[0135] Table 3 and The consistency between the elements with value 1 in

[0136]

[0137] 4.2 Calculate the consistency between the detected visibility label vector and the true visibility level label vector (different weight scaling functions)

[0138] The results of the five scaling functions for the ordinal consistency calculation method of formula (2) are shown in Table 4. On the JS-FHVI dataset, using logarithmic scaling and achieved the best classification effect, far higher than the original ordinal regression method. On the DG-FHVI dataset, using exponential scaling and achieved the best classification effect, 2.65 higher than the original ordinal regression. As Figure 5 shown is and the consistency graph between.

[0139] Table 4 and the consistency between

[0140]

[0141] 4.3 Calculate the visibility label vector of the detection and the true visibility level label vector the consistency of the difference between the part with element 1 and the part with element 0 (different weight scaling functions)

[0142] For the ordinal consistency calculation method of formula (3), this application only considered the method of (Table 5). On the JS-FHVI dataset, using linear scaling achieved the best classification effect, far higher than the original ordinal regression method. On the DG-FHVI dataset, using logarithmic scaling achieved the best classification effect, 1.74 higher than the original ordinal regression. As Figure 6 shown is and the consistency graph of the difference between the part with element 1 and the part with element 0.

[0143] Table 5 and the consistency of the difference between the part with element 1 and the part with element 0

[0144]

[0145] 4.4 The visibility label vector of the detection and the true visibility level label vector IoU (Intersection over Union) of (different weight scaling functions):

[0146] The results of the five scaling functions for the ordinal consistency calculation method of formula (4) are shown in Table 6. On the JS-FHVI dataset, using square root of exponent scaling in and All achieved the best classification results, far higher than the original ordinal regression method. On the DG-FHVI dataset, using exponential square root scaling and also achieved the highest classification accuracy, 3.97 higher than the original ordinal regression. As Figure 7 shown in and is the IoU graph of

[0147] Table 6 and 's IoU

[0148]

[0149] In a feasible implementation manner, as Figure 8 shown is the comparison of the experimental results of the ordinal regression method based on ordinal consistency and the original ordinal regression method at each level on the JS-FHVI and DG-FHVI datasets. Figure 8 The first row is an example of JS-FHVI, and the second row is an example of DG-FHVI. For the examples where the original ordinal regression method detects incorrect visibility levels, the designed method based on ordinal consistency constraints can correct these errors and detect the correct visibility levels.

[0150] In the embodiments of the present invention, a novel visibility detection method based on ordinal consistency constraints is proposed. By converting the traditional visibility level classification task into multiple ordered binary classifiers, the consistency of the classifier results is constrained.

[0151] Meanwhile, an adaptive ordinal consistency constraint is further proposed, and the consistency is used to weight the training samples, increasing the weights of the samples with detection inconsistencies, so that the model focuses on detecting samples with ordinal inconsistencies, improving the accuracy of visibility classification.

[0152] Detailed experiments were conducted on two visibility detection datasets collected from real highway scene datasets. The experimental results show that the visibility detection method based on ordinal consistency constraints proposed in this application can effectively improve the classification accuracy of the visibility detection model.

[0153] Figure 9 is a block diagram of a visibility detection device 300 based on ordinal consistency constraints shown according to an exemplary embodiment. The device 300 is used for the visibility detection method based on ordinal consistency constraints. Referring to Figure 9 , the device includes an image data acquisition module 310, a model construction module 320, a visibility detection module 330, and a detection result calibration module 340. Among them:

[0154] The image data acquisition module 310 is used to acquire image data through a video monitoring device;

[0155] A model construction module 320 for constructing a visibility detection model based on ordinal consistency constraints; the visibility detection model includes two parts: an ordinal regression network based on VGG16 and an ordinal consistency constraint loss function;

[0156] A visibility detection module 330 for inputting image data into the ordinal regression network based on VGG16, extracting visibility features of the image, and obtaining a classification result of visibility level labels;

[0157] A detection result calibration module 340 for obtaining the classification result of visibility level labels, assigning different weight information to each sample, calculating the consistency between the detected visibility label classification result and the true visibility level label through the ordinal consistency constraint loss function, obtaining an accurate visibility level classification result, and completing the visibility detection based on ordinal consistency constraints.

[0158] Optionally, the model construction module 320, the ordinal regression network based on VGG16 includes 13 convolutional network layers, and the ReLU function is used after all convolutional layers; max pooling and batch normalization are used after the first, fourth, seventh, tenth, and last convolutional network layers; learn to extract the mapping of the ordinal relationship between visibility features and visibility levels from highway images through the ordinal regression based on VGG16;

[0159] Ensure the consistency between the ordinal information learned by the visibility detection model and the true value through the ordinal consistency constraint loss.

[0160] Optionally, the visibility detection module 330 for inputting image data into the ordinal regression network based on VGG16;

[0161] Shrink the feature map to a fixed size through the adaptive average pooling layer of the ordinal regression network based on VGG16;

[0162] Extract depth feature information related to visibility through two fully connected layers;

[0163] Obtain the visibility level label corresponding to the image data, and expand the visibility level label into a vector that retains the order information of the level labels;

[0164] Learn the ordinal relationship between the image visibility features and the level labels through ordinal regression, and obtain the classification result of visibility level labels.

[0165] Optionally, obtaining the visibility level label corresponding to the image data and expanding the visibility level label into a vector that retains the order information of the level labels includes:

[0166] Obtain the picture , the corresponding visibility level label is , where W represents the image width, H represents the image height, C represents the number of image channels, i represents the i-th image, and K represents the number of visibility level categories;

[0167] Encode the visibility level label into a vector with k elements .

[0168] Optionally, learn the ordinal relationship between the image visibility features and the level labels through ordinal regression to obtain the visibility level label classification result, including:

[0169] The k-th element The calculation method is as shown in the following formula (1):

[0170] (1)

[0171] where ;

[0172] Use classifiers to map the deep visibility feature embeddings from two fully connected layers to a binary output visibility label vector consistent with the visibility level label ;

[0173] Calculate the sum of the outputs of the classifiers to obtain the final detected visibility level of the picture as shown in the following formula (2):

[0174] (2).

[0175] Optionally, the detection result calibration module 340 is used to define an ordinal consistency constraint loss function as shown in the following formula (3):

[0176] (3)

[0177] where N represents the number of training set images, represents the exponential open scaling function, represents the ordinal consistency result, represents the cross-entropy loss function;

[0178] Define the exponential square root scaling function as shown in the following formula (4):

[0179] (4)

[0180] For the calculated ordinal consistency result Use the exponential square root scaling function as the weight value assigned to the samples;

[0181] Define the cross-entropy function as the following formula (5):

[0182] (5)

[0183] where represents the number of visibility level categories, represents an indicator function that results in 1 when the condition is true and 0 when the condition is false; p represents the model prediction probability; represents the th image; represents the parameters of the model;

[0184] Obtain the visibility label vector and the true visibility label vector ;

[0185] By calculating the intersection over union of the detected visibility label vector and the true visibility level label vector obtain the ordinal consistency result of the visibility level; and use the exponential square root scaling function to assign higher weights to the difficult samples with ordinal inconsistencies, so that the model focuses on the difficult samples;

[0186] Obtain an accurate visibility level classification result and complete the visibility detection based on the ordinal consistency constraint.

[0187] Optionally, by calculating the intersection over union of the detected visibility label vector and the true visibility level label vector obtain the ordinal consistency result of the visibility level, including:

[0188] Calculate the ordinal consistency of the visibility level according to the following formula (6):

[0189] (6).

[0190] In an embodiment of the present invention, a visibility detection method based on ordinal consistency constraint is proposed. By converting the traditional visibility level classification task into multiple ordered binary classifiers, the consistency of the classifier results is constrained. At the same time, an adaptive ordinal consistency constraint is further proposed, and the consistency is used to weight the training samples, increasing the weights of the detection-inconsistent samples and improving the accuracy of visibility classification. This method is trained and tested on the image dataset collected by video monitoring devices on highways, and the ablation experiment is used to verify the effect of this method. The experimental results show that the visibility detection method based on ordinal consistency constraint proposed in this application can effectively improve the classification accuracy of the visibility detection model.

[0191] Figure 10 FIG. is a schematic structural diagram of a visibility detection device based on ordinal consistency constraint provided by an embodiment of the present invention, as Figure 10 shown, the visibility detection device based on ordinal consistency constraint may include the above-mentioned Figure 9 visibility detection device based on ordinal consistency constraint shown. Optionally, the visibility detection device 410 based on ordinal consistency constraint may include a first processor 2001.

[0192] Optionally, the visibility detection device 410 based on ordinal consistency constraint may further include a memory 2002 and a transceiver 2003.

[0193] Among them, the first processor 2001 is connected to the memory 2002 and the transceiver 2003, such as through a communication bus.

[0194] Next, in combination with Figure 10 each component of the visibility detection device 410 based on ordinal consistency constraint will be specifically introduced:

[0195] Among them, the first processor 2001 is the control center of the visibility detection device 410 based on ordinal consistency constraint, which may be a processor or a collective term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or may be a specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as: one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs).

[0196] Optionally, the first processor 2001 may execute various functions of the visibility detection device 410 based on ordinal consistency constraints by running or executing software programs stored in the memory 2002 and invoking data stored in the memory 2002.

[0197] In a specific implementation, as an example, the first processor 2001 may include one or more CPUs, such as Figure 10 CPU0 and CPU1 shown in

[0198] In a specific implementation, as an example, the visibility detection device 410 based on ordinal consistency constraints may also include multiple processors, such as Figure 10 the first processor 2001 and the second processor 2004 shown in

[0199] Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, the processor may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0200] The memory 2002 is used to store software programs for implementing the solution of the present invention and is controlled by the first processor 2001 for execution. The specific implementation manner may refer to the above method embodiments and will not be elaborated herein.

[0200] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001 or exist independently and be coupled to the first processor 2001 through an interface circuit ( Figure 10 not shown in

[0201] The transceiver 2003 is used to communicate with network devices or terminal devices.

[0202] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 10 not shown separately). Among them, the receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0203] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently and be coupled to the first processor 2001 through an interface circuit ( Figure 10 not shown) of the visibility detection device 410 based on ordinal consistency constraints. The embodiments of the present invention do not make specific limitations on this.

[0204] It should be noted that Figure 10 the structure of the visibility detection device 410 based on ordinal consistency constraints shown in

[0205] does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements.

[0206] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), and this processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or this processor may also be any conventional processor, etc.

[0207] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0208] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware, or any combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable sensors. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that contains one or more collections of available media. The available media can be magnetic media (such as floppy disks, hard disks, magnetic tapes), optical media (such as DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0209] It should be understood that the term "and / or" in this application is merely a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Here, A and B can be singular or plural. In addition, the character " / " in this application generally represents an "or" relationship between the associated objects before and after, but it may also represent an "and / or" relationship, which can be specifically understood with reference to the context.

[0210] It should be understood that in various embodiments of the present invention, the magnitudes of the serial numbers of the above processes do not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0211] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0212] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or may be distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0213] In addition, in each embodiment of the present invention, each functional unit may be integrated in a processing unit, may exist physically separately for each unit, or two or more units may be integrated in one unit.

[0214] If the said function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present invention.

[0215] As mentioned above, it is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the said claims.

Claims

1. A visibility detection method based on ordinal consistency constraint, characterized in that: The method comprises: S1. Collect image data through video surveillance equipment; S2. Construct a visibility detection model based on ordinal consistency constraint; the visibility detection model includes two parts: an ordinal regression network based on VGG16 and an ordinal consistency constraint loss function; S3, inputting the image data into the VGG16-based ordinal regression network, extracting the visibility features of the image, and obtaining the visibility level label classification result; S4. Obtain the visibility level label classification results, assign different weight information to each sample, calculate the consistency of the detected visibility label classification results and the true visibility level label by the intersection and union ratio based on the ordinal consistency constraint loss function, obtain accurate visibility level classification results, and complete the visibility detection based on the ordinal consistency constraint.

2. The visibility detection method based on ordinal consistency constraint according to claim 1, characterized in that S2 include: The VGG16-based ordinal regression network includes 13 convolutional network layers, and ReLU functions are used after all convolutional layers; maximum pooling and batch normalization are used after the first, fourth, seventh, tenth and last convolutional network layers; Learning to extract a mapping of the ordinal relationship between visibility features and visibility levels from highway images through the VGG16-based ordinal regression; The consistency between the ordinal information learned by the visibility detection model and the true value is ensured by the ordinal consistency constraint loss.

3. The visibility detection method based on ordinal consistency constraint according to claim 2 is characterized in that: In S3, the image data is input into the VGG16-based ordinal regression network to extract the visibility features of the image and obtain the visibility level label classification result, including: Inputting the image data into the VGG16-based ordinal regression network; The feature map is reduced to a fixed size by an adaptive average pooling layer of the VGG16-based ordinal regression network; The deep feature information related to visibility is extracted through two fully connected layers; Obtaining a visibility level label corresponding to the image data, and expanding the visibility level label into a vector that retains level label order information; The ordinal relationship between image visibility features and grade labels is learned through ordinal regression to obtain the visibility grade label classification results.

4. The visibility detection method based on ordinal consistency constraint according to claim 3 is characterized in that: The step of obtaining the visibility level label corresponding to the image data and expanding the visibility level label into a vector that retains level label sequence information includes: Get the image , then the corresponding visibility level label is , where W represents the image width, H represents the image height, C represents the number of image channels, i represents the i-th image, and K represents the number of visibility level categories; Label the visibility level Encoded as a vector with k elements .

5. The visibility detection method based on ordinal consistency constraint according to claim 6 is characterized in that: The ordinal relationship between the image visibility feature and the grade label is learned through ordinal regression to obtain the visibility grade label classification result, including: The kth element The calculation method is as follows: （1） in, ; use A classifier maps the deep visibility feature embeddings from the two fully connected layers to a binary output visibility label vector consistent with the visibility class label ; calculate The sum of the classifier outputs gives the image The final detection visibility level is as follows: (2)。 6. The visibility detection method based on ordinal consistency constraint according to claim 5 is characterized in that: In S4, the visibility level label classification result is obtained, and different weight information is assigned to each sample. The consistency is calculated by the intersection and union ratio of the detected visibility label classification result and the real visibility level label based on the ordinal consistency constraint loss function to obtain an accurate visibility level classification result, and the visibility detection based on the ordinal consistency constraint is completed, including: Define the ordinal consistency constraint loss function As shown in the following formula (3): （3） Where N represents the number of images in the training set. represents the exponential open scaling function, represents the ordinal consistency result, represents the cross entropy loss function; Define the exponential scaling function As shown in the following formula (4): （4） The calculated ordinal consistency results Use the exponential root scaling function as the weight value assigned to the sample; The cross entropy function is defined as follows: （5） in, Indicates the visibility level category number, represents an indicator function, the result is 1 when the condition is true, and the result is 0 when the condition is false; p represents the model prediction probability; Indicates images; Represents the parameters of the model; Get visibility label vector and the true visibility label vector ; By calculating the visibility label vector of the detection and the true visibility level label vector The intersection-and-union ratio is calculated to obtain the ordinal consistency result of the visibility level; and the exponential square root scaling function is used to give higher weights to difficult samples with inconsistent ordinals, so that the model can focus on difficult samples; Obtain accurate visibility grade classification results and complete visibility detection based on ordinal consistency constraints.

7. The visibility detection method based on ordinal consistency constraint according to claim 6 is characterized in that: The visibility label vector detected by calculating and the true visibility level label vector The intersection and union ratio of is used to obtain the ordinal consistency results of visibility levels, including: The ordinal consistency of visibility levels is calculated according to the following formula (6): （6）。 8. A visibility detection device based on ordinal consistency constraint, the visibility detection device based on ordinal consistency constraint is used to implement the visibility detection method based on ordinal consistency constraint as claimed in any one of claims 1 to 7, characterized in that: The device comprises: An image data acquisition module, used to acquire image data through a video surveillance device; A model building module, used to build a visibility detection model based on ordinal consistency constraints; the visibility detection model includes two parts: an ordinal regression network based on VGG16 and an ordinal consistency constraint loss function; A visibility detection module, used to input the image data into the VGG16-based ordinal regression network, extract the visibility features of the image, and obtain a visibility level label classification result; The detection result calibration module is used to obtain the visibility level label classification results, assign different weight information to each sample, and calculate the consistency of the intersection and union ratio of the detected visibility label classification results and the true visibility level labels based on the ordinal consistency constraint loss function to obtain accurate visibility level classification results and complete the visibility detection based on the ordinal consistency constraint.

9. A visibility detection device based on ordinal consistency constraints, the visibility detection device based on ordinal consistency constraints comprising: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, any one of the visibility detection methods based on ordinal consistency constraints as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the visibility detection methods based on ordinal consistency constraints as described in any one of claims 1 to 7.

Images