A visibility detection method and device based on ordinal consistency constraint

Abstract

The present invention relates to the field of visibility detection technology, and in particular to a visibility detection method and device based on ordinal consistency constraints. The method comprises: constraining the consistency of the classifier results by converting the traditional visibility level classification task into a plurality of ordered binary classifiers. At the same time, an adaptive ordinal consistency constraint is further proposed, and the consistency is used to weight the training samples, increase the weight of the inconsistent detection samples, and improve the accuracy of visibility classification. The method is trained and tested on an image dataset collected by video surveillance equipment on a highway, and the effect of the method is verified by an ablation experiment. The experimental results show that the visibility detection method based on ordinal consistency constraints proposed in this application can effectively improve the classification accuracy of the visibility detection model.

Description

Technical Field

[0001] The present invention relates to the technical field of visibility detection, and in particular to a visibility detection method and device based on ordinal consistency constraints. Background Art

[0002] Visibility, as a key indicator of atmospheric transparency, plays a vital role in environmental monitoring, traffic management, and public safety. Traditional visibility measurement methods, such as transmissive visibility meters, while highly accurate, suffer from limitations such as high cost and difficulty in deploying measurement points, making large-scale, high-density monitoring difficult. With the widespread adoption of video surveillance technology and the rapid development of image processing algorithms, visibility inversion techniques based on video images have become a new research hotspot. Numerous studies have been conducted in China on this type of visibility inversion technology. For example, a study proposed a low visibility recognition algorithm based on real-scene image conversion and convolutional neural network [Liu Dongwei, Mu Haizhen, He Qianshan, et al. A low visibility recognition algorithm based on real-scene images. Journal of Applied Meteorology, 2022, 33(4): 501-512. LIU DW, MU HZ, HE QS, et al. A low visibility recognition algorithm based on surveillance video [J]. Journal of applied meteorological science, 2022, 33(4): 501-512.], which achieved effective classification of visibility levels by analyzing the gradient, saturation and brightness information of the image. In addition, satellite remote sensing technology has also been used to invert ground visibility. By combining the MODIS satellite aerosol optical depth and the aerosol elevation field simulated by GEOS-Chem, a visibility inversion algorithm was proposed [Zhang Yan, Li Jing. Research on the algorithm for inverting ground visibility based on satellite aerosol optical depth [J]. Journal of Peking University (Natural Science Edition), 2020, 56 (02): 231-241. Zhang Y, LiJ. Retrieval of surface visibility using satellite - based aerosolmeasurements. Acta Scientiarum Naturalium Universitatis Pekinensis, 2020, 56(2): 231-241.].Some studies have proposed traffic visibility estimation models based on different scenarios [Liu Qiyang, Qiao Fengxue, Chen Bo, et al., 2022. Application of traffic visibility estimation models in different scenarios [J]. Journal of Atmospheric Sciences, 45(2): 179-190. Analysis of the application of traffic visibility estimation models in different scenarios[J]. Trans Atmos Sci,45( 2) : 179-190.]. These models make comprehensive use of meteorological observation data and image features, effectively reducing the dependence on observation instruments and improving the flexibility and efficiency of visibility monitoring.

[0003] In recent years, with the rapid development of deep learning technology, visibility detection models based on deep learning models have achieved good performance and have attracted widespread attention from researchers. Some researchers regard visibility estimation as a regression task. Li et al. [Li Q, Xie B. Visibility estimation using a single image[C] / / ComputerVision: Second CCF Chinese Conference, CCCV 2017, Tianjin, China, October 11–14, 2017, Proceedings, Part I. Springer Singapore, 2017: 343-355.] Assuming that the extinction coefficient of the haze-free image is approximately constant, combined with the dark channel prior [He K, Sun J, Tang X. Single image hazeremoval using dark channel prior[J]. IEEE transactions on pattern analysisand machine intelligence, 2010, 33(12): 2341-2353], the ratio of the two extinction coefficients in the hazy image and the haze-free image can be calculated. On this basis, the extinction coefficient of the hazy image is calculated and combined with Koschmieder's law [Koschmieder H. Theorie der horizontalen Sichtweite[M]. Keim & Nemnich, 1925] and the contrast decay law were used to calculate visibility on real urban scene images. Subsequently, a visibility detection method was further improved by using transmittance optimization based on edge degradation [Li Q, Li Y, Xie B. Single image-based scene visibility estimation[J]. Ieee Access, 2019, 7: 24430-24439]. This method assumes that the extinction coefficient of fog-free images is approximately constant. The visibility calculation based on atmospheric physics theory has large errors, resulting in inaccurate visibility estimates.You[You Y, Lu C, Wang W, et al. Relative CNN-RNN:Learning relative atmospheric visibility from images[J]. IEEE Transactions onImage Processing, 2018, 28(1): 45-55] and others directly estimated the image visibility value. This method constructs a CNN-RNN model and combines it with support vector machine SVM to learn the relative visibility of image pairs, thereby estimating absolute visibility. This method considers the ordinal relationship between the visibility of image pairs for the first time, but it requires a lot of data preparation in the early stage and cannot achieve end-to-end training. Lo[Lo WL, Zhu M, Fu H. Meteorology visibility estimation byusing multi-support vector regression method[J]. Journal of Advances inInformation Technology Vol, 2020, 11(2): 40-47] and others proposed a sub-region visibility estimation method. This method selects valid subregions from images of landmark objects with different visibility ranges, uses support vector machines (SVM) for feature classification, and directly estimates visibility using multi-head support vector regression (MSVR) representing different visibility ranges. Later, Lo et al. [Li J, Lo WL, Fu H, et al. A transfer learning method for meteorological visibility estimation based on feature fusion method[J]. Applied Sciences, 2021, 11(3): 997.] improved this method: using grayscale weighted averaging to determine multiple valid subregions of the image to estimate the visibility of each subregion, and then using weighted fusion to estimate the visibility of the entire image, further reducing the visibility estimation error.Xun et al. [Xun L, Zhang H, Yan Q, et al. VISOR-NET: visibility estimation based on deep ordinal relative learning underdiscrete-level labels[J]. Sensors, 2022, 22(16): 6227] proposed a method for visibility level classification and visibility estimation using the ordinal information and relative relationship of the image. This method only achieved visibility level classification on the collected highway images. Yao [Yao S, Huang B. Extraction of aerosol opticalextinction properties from a smartphone photograph to measure visibility[J].IEEE Transactions on Geoscience and Remote Sensing, 2021, 60: 1-13] et al. extracted AOEPs (i.e., local medium transmission rate and local medium extinction coefficient) from the image and combined them with atmospheric physics theory to directly estimate visibility. This method requires detecting the vanishing point of the image and selecting appropriate AOEPs to regress visibility. Due to the influence of lighting, obstructions, camera viewing angle, etc., the detection of the vanishing point is difficult. You[You J, Jia S, Pei X, et al.DMRVisNet: Deep multihead regression network for pixel-wise visibility estimation under foggy weather[J]. IEEE Transactions on IntelligentTransportation Systems, 2022, 23(11): 22354-22366] et al. used an encoder and three decoders to extract air light, transmittance, and depth information, and combined Koschmieder's law to calculate pixel-level visibility maps and image-level visibility on the synthesized image. This method can be applied to real road scenes for visibility estimation.In the same year, Zhang et al. [Zhang F, Yu T, Li Z, et al. Deep quantified visibility estimation for traffic image[J]. Atmosphere, 2022, 14(1): 61.] proposed an end-to-end deep quantized visibility estimation network consisting of a transmittance estimation module, a depth estimation module, and an extinction collaborative estimation module. This method achieves end-to-end direct visibility estimation in highway images based on atmospheric physics methods.

[0004] In addition to treating visibility estimation as a regression task, some researchers have treated visibility estimation as a classification task, learning the relationship between image features and visibility levels. In 2017, Li et al. [Li S, Fu H, Lo W L.Meteorological visibility evaluation on webcam weather image using deeplearning features[J]. Int. J. Comput. Theory Eng, 2017, 9(6): 455-461] first used CNN to estimate the visibility of road images captured by cameras. Based on a large-scale fog image dataset and visibility annotations, they used AlexNet [Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.] to extract visibility features from images and perform visibility level classification, achieving good application results. Vaibhav et al. [Vaibhav V, Konda KR, Kondapalli C, et al. Real-time fog visibility range estimation for autonomous driving applications[C] / / 2020IEEE 23rd International Conference on Intelligent Transportation Systems(ITSC). IEEE, 2020: 1-6.] combined image preprocessing techniques to extract block discrete cosine transform (DCT) features and block-level Shannon entropy features, and input them into a CNN module along with the original image features to achieve visibility level classification.Unlike traditional deep learning methods that only use convolutional neural networks (CNNs) to process input images, Liu et al. [Liu J, Chang X, Li Y, et al. STCN-Net: A novel multi-feature stream fusion visibility estimation approach[J]. IEEE Access, 2022, 10: 120329-120342] combined CNN and Transformer [Liu Z, Lin Y, Cao Y, et al. Swin transformer: Hierarchical vision transformer using shifted windows[C] / / Proceedings of the IEEE / CVFinternational conference on computer vision. 2021: 10012-10022] to process input images and proposed an STCN-Net for estimating visibility levels. This method designs a new 3D multi-feature flow matrix DDT for visibility estimation, which consists of a transmittance matrix, a dark channel matrix, and a depth matrix. Later, Liu et al. [Liu J, Zhong J, Li Y, et al. Fgs-net: A visibility estimation method based on statistical feature stream in fog area[J]. 2023] improved the model and proposed a new fog region segmentation method to segment the fog region in the input image. Subsequently, the DDT matrix, contrast, average gradient, and brightness features were extracted from the fog region and combined with the learning features extracted by Transformer for visibility level classification, achieving good visibility classification performance. However, the existing deep learning model ignores the order relationship between visibility levels. The ordinal regression network [Liu H, Lu J, Feng J, et al. Ordinal deep learning for facial age estimation[J]. IEEE transactions on circuits and systems for video technology, 2017, 29(2): 486-501.] extends the classifier to multiple binary classifiers for ordered regression, and better captures visibility-related features by utilizing the ordinal relationship between data.However, the ordinal regression network has the problem of inconsistent detection results of multiple binary classifiers, which leads to unstable detection results of the model. Summary of the Invention

[0005] In order to solve the technical problem in the prior art that the detection results of multiple binary classifiers in the ordinal regression network are inconsistent, resulting in unstable detection results of the model, the embodiment of the present invention provides a visibility detection method and device based on ordinal consistency constraints. The technical solution is as follows:

[0006] In one aspect, a visibility detection method based on ordinal consistency constraint is provided, characterized in that the method comprises:

[0007] S1. Collect image data through video surveillance equipment;

[0008] S2. Construct a visibility detection model based on ordinal consistency constraint; the visibility detection model includes two parts: an ordinal regression network based on VGG16 and an ordinal consistency constraint loss function;

[0009] S3. Input the image data into the VGG16-based ordinal regression network to extract the visibility features of the image and obtain the visibility level label classification result;

[0010] S4. Obtain the visibility level label classification results, assign different weight information to each sample, calculate the consistency of the detected visibility label classification results and the true visibility level label by the intersection and union ratio based on the ordinal consistency constraint loss function, obtain accurate visibility level classification results, and complete visibility detection based on ordinal consistency constraints.

[0011] Optionally, S2 includes: a VGG16-based ordinal regression network comprising 13 convolutional network layers, wherein ReLU functions are used after all convolutional layers; maximum pooling and batch normalization are used after the first, fourth, seventh, tenth, and last convolutional network layers; and learning a mapping between visibility features and ordinal relationships between visibility levels extracted from highway images through VGG16-based ordinal regression.

[0012] The ordinal consistency constraint loss is used to ensure the consistency between the ordinal information learned by the visibility detection model and the true value.

[0013] Optionally, in S3, the image data is input into an ordinal regression network based on VGG16 to extract the visibility features of the image and obtain a visibility level label classification result, including:

[0014] Input the image data into the VGG16-based ordinal regression network;

[0015] The feature map is reduced to a fixed size through an adaptive average pooling layer of a VGG16-based ordinal regression network;

[0016] Extract deep feature information related to visibility through two fully connected layers;

[0017] Obtain visibility level labels corresponding to the image data, and expand the visibility level labels into vectors that retain the order information of the level labels;

[0018] The ordinal relationship between image visibility features and grade labels is learned through ordinal regression to obtain the visibility grade label classification results.

[0019] Optionally, obtaining a visibility level label corresponding to the image data and expanding the visibility level label into a vector that retains the level label order information includes:

[0020] Get the image , then the corresponding visibility level label is , where W represents the image width, H represents the image height, C represents the number of image channels, i represents the i-th image, and K represents the number of visibility level categories;

[0021] Label the visibility level Encoded as a vector with k elements .

[0022] Optionally, an ordinal relationship between image visibility features and grade labels is learned through ordinal regression to obtain a visibility grade label classification result, including:

[0023] kth element The calculation method is as follows:

[0024] (1)

[0025] in, ;

[0026] use A classifier maps the deep visibility feature embeddings from the two fully connected layers to a binary output visibility label vector consistent with the visibility level label. ;

[0027] calculate The sum of the classifier outputs is used to get the image The final detection visibility level is as follows:

[0028] (2).

[0029] Optionally, obtain the visibility level label classification result, assign different weight information to each sample, and calculate the consistency of the detected visibility label classification result and the true visibility level label by the intersection-over-union ratio based on the ordinal consistency constraint loss function to obtain an accurate visibility level classification result, thereby completing the visibility detection based on the ordinal consistency constraint, including:

[0030] Define the ordinal consistency constraint loss function As shown in the following formula (3):

[0031] (3)

[0032] Where N represents the number of images in the training set. represents the exponential open scaling function, represents the ordinal consistency result, represents the cross entropy loss function.

[0033] Define the exponential scaling function As shown in the following formula (4):

[0034] (4)

[0035] The calculated ordinal consistency results Use the exponential scaling function as the weight value assigned to the sample;

[0036] The cross entropy function is defined as follows:

[0037] (5)

[0038] in, Indicates the number of visibility level categories, represents an indicator function, the result is 1 when the condition is true and 0 when the condition is false; p represents the model prediction probability; Indicates the images; Represents the parameters of the model;

[0039] Get visibility label vector and the true visibility label vector ;

[0040] By calculating the visibility label vector of the detection and the true visibility level label vector The intersection-over-union ratio is used to obtain the ordinal consistency result of the visibility level; and the exponential square root scaling function is used to give higher weights to difficult samples with inconsistent ordinals, so that the model can focus on difficult samples;

[0041] Obtain accurate visibility level classification results and complete visibility detection based on ordinal consistency constraints.

[0042] Optionally, by computing the visibility label vector of the detections and the true visibility level label vector The ordinal consistency results of visibility levels are obtained by using the intersection-union ratio of , including:

[0043] The ordinal consistency of visibility levels is calculated according to the following formula (6):

[0044] (6).

[0045] On the other hand, a visibility detection device based on ordinal consistency constraint is provided, which is applied to the visibility detection method based on ordinal consistency constraint, and includes:

[0046] An image data acquisition module, used to acquire image data through video surveillance equipment;

[0047] A model building module is used to build a visibility detection model based on ordinal consistency constraints. The visibility detection model consists of two parts: an ordinal regression network based on VGG16 and an ordinal consistency constraint loss function.

[0048] The visibility detection module is used to input image data into the VGG16-based ordinal regression network to extract the visibility features of the image and obtain the visibility level label classification results;

[0049] The detection result calibration module is used to obtain the visibility level label classification results, assign different weight information to each sample, and calculate the consistency of the intersection and union ratio of the detected visibility label classification results and the true visibility level labels based on the ordinal consistency constraint loss function to obtain accurate visibility level classification results and complete visibility detection based on ordinal consistency constraints.

[0050] On the other hand, a visibility detection device based on ordinal consistency constraints is provided, and the visibility detection device based on ordinal consistency constraints includes: a processor; a memory, wherein computer-readable instructions are stored on the memory, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned visibility detection methods based on ordinal consistency constraints is implemented.

[0051] On the other hand, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned visibility detection methods based on ordinal consistency constraints.

[0052] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0053] In this embodiment of the present invention, a novel visibility detection method based on ordinal consistency constraints is proposed. This method transforms the traditional visibility level classification task into multiple ordered binary classifiers to constrain the consistency of the classifier results. Furthermore, an adaptive ordinal consistency constraint is proposed, and consistency is used to weight training samples, increasing the weight of inconsistent samples and improving the accuracy of visibility classification. Detailed experiments on a visibility detection dataset demonstrate that the proposed method can effectively improve the classification accuracy of the visibility detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0055] Figure 1 A flowchart of a visibility detection method based on ordinal consistency constraints provided by an embodiment of the present invention;

[0056] Figure 2 A network diagram of visibility level classification based on ordinal consistency provided by an embodiment of the present invention;

[0057] Figure 3 Example diagrams of all levels of the JS-FHVI and DG-FHVI datasets provided in the embodiments of the present invention;

[0058] Figure 4 The embodiment of the present invention provides and The consistency graph between the parts whose elements are 1;

[0059] Figure 5 The embodiment of the present invention provides and The consistency diagram between

[0060] Figure 6 The embodiment of the present invention provides and A consistency graph of the difference between the part where the element is 1 and the part where the element is 0;

[0061] Figure 7 The embodiment of the present invention provides and IoU graph;

[0062] Figure 8 Comparison chart of experimental results provided by embodiments of the present invention;

[0063] Figure 9 A block diagram of a visibility detection device based on ordinal consistency constraints provided by an embodiment of the present invention;

[0064] Figure 10 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0066] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0067] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0068] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0069] The embodiment of the present invention provides a visibility detection method based on ordinal consistency constraints, which can be implemented by a visibility detection device based on ordinal consistency constraints, and the visibility detection device based on ordinal consistency constraints can be a terminal or a server. Figure 1 The flow chart of the visibility detection method based on ordinal consistency constraint is shown in FIG. Figure 1 As shown, the visibility detection method based on ordinal consistency constraint proposed by the present invention may include the following steps:

[0070] S1. Collect image data through video surveillance equipment.

[0071] S2. Construct a visibility detection model based on ordinal consistency constraints. The visibility detection model consists of two parts: an ordinal regression network based on VGG16 (Visual Geometry Group 16-layer network) and an ordinal consistency constraint loss function.

[0072] In a feasible implementation, there is a natural ordinal relationship between visibility levels, and learning this ordinal relationship based on image visibility features can achieve visibility level classification. Therefore, the present application first constructs a visibility detection model based on ordinal regression. However, for difficult samples where it is difficult to learn the correct ordinal relationship, the model may have a problem of inconsistency between the learned ordinal relationship and the true ordinal information, thereby confusing the learning ability of the model, resulting in an inability to effectively learn the correct ordinal information from the extracted visibility features, thereby reducing the accuracy of visibility level classification. In order to solve this problem, the present application further designs a visibility detection model based on ordinal consistency constraints. The overall framework diagram is shown as follows. Figure 2 As shown in the figure, the model constrains the ordinal information learned by the model through adaptive ordinal consistency constraint loss, so that the model can focus on difficult samples with ordinal inconsistency problems, better learn and process those samples with ordinal inconsistency problems, and thus improve the performance of visibility classification.

[0073] In one feasible implementation, the ordinal regression network based on VGG16 contains 13 convolutional network layers, and the ReLU function is used after all convolutional layers;

[0074] Max pooling and batch normalization are used after the first, fourth, seventh, tenth, and last convolutional network layers to reduce image dimensionality;

[0075] We learn to extract the mapping between visibility features and the ordinal relationship between visibility levels from highway images through ordinal regression based on VGG16.

[0076] The ordinal consistency constraint loss is used to ensure the consistency between the ordinal information learned by the visibility detection model and the true value.

[0077] S3. Input the image data into the VGG16-based ordinal regression network to extract the visibility features of the image and obtain the visibility level label classification result;

[0078] In a feasible implementation, in S3, the image data is input into an ordinal regression network based on VGG16 to extract the visibility features of the image and obtain the visibility level label classification result, including:

[0079] Input the image data into the VGG16-based ordinal regression network;

[0080] The feature map is reduced to a fixed size through an adaptive average pooling layer of a VGG16-based ordinal regression network;

[0081] Extract deep feature information related to visibility through two fully connected layers;

[0082] Obtain the visibility level labels corresponding to the image data, and expand the visibility level labels into vectors that retain the order information of the level labels. Use ordinal regression to learn the ordinal relationship between image visibility features and level labels, thereby realizing visibility level label classification.

[0083] The ordinal relationship between image visibility features and grade labels is learned through ordinal regression to obtain the visibility grade label classification results.

[0084] In a feasible implementation, obtaining the visibility level labels corresponding to the image data and expanding the visibility level labels into vectors that retain the order information of the level labels include:

[0085] Get the image , then the corresponding visibility level label is , where W represents the image width, H represents the image height, C represents the number of image channels, i represents the i-th image, and K represents the number of visibility level categories;

[0086] Label the visibility level Encoded as a vector with k elements .

[0087] In a feasible implementation, ordinal regression is used to learn the ordinal relationship between image visibility features and grade labels to obtain visibility grade label classification results, including:

[0088] kth element The calculation method is as follows:

[0089] (1)

[0090] in, ;

[0091] use A classifier maps the deep visibility feature embeddings from the two fully connected layers to a binary output visibility label vector consistent with the visibility level label. ;

[0092] calculate The sum of the classifier outputs is used to get the image The final detection visibility level is as follows:

[0093] (2).

[0094] In one possible implementation, relying solely on ordinal regression may cause problems when dealing with difficult samples, especially when the ordinal relationship learned by the model is inconsistent with the true ordinal, for example, the visibility level of an image of a highway , after label expansion , however, the output of the model is , then the picture Detection visibility level = 3. Although the final visibility level and true visibility Same, but and This ordinal inconsistency can confuse the model's learning process. To address this issue, this application proposes an ordinal consistency constraint loss function. By using adaptive ordinal consistency constraints, different weight information is assigned to each sample, allowing the model to focus on detecting samples with inconsistent ordinal relationships:

[0095] S4. Obtain the visibility level label classification results, assign different weight information to each sample, calculate the consistency of the detected visibility label classification results and the true visibility level label by the intersection and union ratio based on the ordinal consistency constraint loss function, obtain accurate visibility level classification results, and complete visibility detection based on ordinal consistency constraints.

[0096] In one feasible implementation, the visibility level label classification result is obtained, and the consistency is calculated by the intersection-over-union ratio of the detected visibility label classification result and the true visibility level label based on the ordinal consistency constraint loss function. Different weight information is assigned to each sample using an exponential square root scaling function to obtain an accurate visibility level classification result, thereby completing the visibility detection based on the ordinal consistency constraint, including:

[0097] Define the ordinal consistency constraint loss function , which aims to minimize the error caused by misclassification of a given input image as shown in the following formula (3):

[0098] (3)

[0099] Where N represents the number of images in the training set. represents the exponential open scaling function, represents the ordinal consistency result, represents the cross entropy loss function.

[0100] Define the exponential scaling function As shown in the following formula (4):

[0101] (4)

[0102] The calculated ordinal consistency results Use the exponential scaling function as the weight value assigned to the sample;

[0103] The cross entropy function is defined as follows:

[0104] (5)

[0105] in, Indicates the number of visibility level categories, represents an indicator function, the result is 1 when the condition is true and 0 when the condition is false; p represents the model prediction probability; Indicates the images; Represents the parameters of the model;

[0106] Get visibility label vector and the true visibility label vector ;

[0107] By calculating the visibility label vector of the detection and the true visibility level label vector The intersection-over-union ratio is used to obtain the ordinal consistency result of the visibility level; and higher weights are given to difficult samples with inconsistent ordinals, so that the model can focus on difficult samples and improve the model's visibility level classification accuracy;

[0108] Obtain accurate visibility level classification results and complete visibility detection based on ordinal consistency constraints.

[0109] In one possible implementation, the visibility label vector of the detection is calculated and the true visibility level label vector The ordinal consistency results of visibility levels are obtained by using the intersection-union ratio of , including:

[0110] The ordinal consistency of visibility levels is calculated according to the following formula (6):

[0111] (6).

[0112] In a feasible implementation manner, the present application provides a detailed description of the above technical solution through the following actual experimental process:

[0113] We first introduce two datasets collected from real highways: JS-FHVI (visibility images of foggy highway No. 1) and DG-FHVI (visibility images of foggy highway No. 2). We then conduct validation and ablation experiments on the proposed method on these two datasets.

[0114] (1) Visibility detection dataset:

[0115] JS-FHVI: 7799 images were collected from 216 video surveillance devices on Highway 1. These images were randomly divided into training, validation, and test sets with ratios of 70%, 10%, and 20%, respectively. Detailed information on each level of the images is shown in Tables 1 and Figure 3 .

[0116] DG-FHVI: This dataset collects image data from 133 video surveillance devices on Highway 2, totaling 6,777 highway images. This dataset is closer to the coast and has different climatic conditions than JS-FHVI. In the experiment, these images were randomly divided into training, validation, and test sets, with ratios of 80%, 10%, and 10%, respectively. Detailed information on each level of the image is shown in Tables 1 and Figure 3 .

[0117] Table 1 Dataset information

[0118]

[0119] (2) Evaluation indicators

[0120] In order to evaluate the accuracy of visibility detection, the accuracy (Acc) commonly used in classification tasks is used as the evaluation indicator in the experiment. It represents the proportion of correctly detected samples to all test samples.

[0121] Four methods of calculating ordinal consistency are defined in the experiment. Formula (6.1) calculates the visibility label vector of the detection and the true visibility level label vector The ordinal consistency of the position where the element is 1 focuses on the position information that determines the visibility level label. Formula (6.2) calculates the visibility label vector of the detection and the true visibility level label vector The ordinal consistency of all elements focuses on the influence of all elements in the vector on the ordinal information. The visibility label vector detected in formula (6.3) The difference between the position where the element is 1 and the position where the element is 0 and the true visibility level label vector The closer the difference is, the higher the ordinal consistency of visibility level is. Formula (6) calculates the visibility label vector of the detection and the true visibility level label vector The higher the intersection-and-union ratio, the higher the ordinal consistency of the visibility level.

[0122] (6.1)

[0123] (6.2)

[0124] (6.3)

[0125] (6)

[0126] (3) Visibility test results

[0127] The detection accuracy of the proposed method on the two datasets is shown in Table 2. It can be seen that the performance of the proposed method is significantly improved compared with the original ordinal regression.

[0128] Table 2 Detection accuracy of different methods

[0129]

[0130] (4) Ablation experiment

[0131] This application compares the effects of five scaling functions on four ordinal consistency losses: linear scaling, exponential scaling, exponential root scaling, logarithmic scaling, and softmax scaling. Furthermore, two weighting methods are also compared: (reduce the weight of hard examples) and (increasing the weight of difficult samples).

[0132] The result is as follows:

[0133] 4.1. Computing the visibility label vector of the detection and the true visibility level label vector The consistency between the parts with elements of 1 (different weight scaling functions)

[0134] From the results in Table 3, we can see that for the ordinal consistency calculation method of formula (1), exponential scaling and exponential square root scaling use and Good classification results are achieved on both datasets. Linear scaling is used on JS-FHVI The highest classification accuracy was achieved. and The consistency between the parts with elements 1 is as follows Figure 4 shown.

[0135] Table 3 and The consistency between the parts where the element is 1

[0136]

[0137] 4.2 Calculating the visibility label vector of the detection and the true visibility level label vector Consistency between (different weight scaling functions)

[0138] The results of the five scaling functions for the ordinal consistency calculation method of formula (2) are shown in Table 4. On the JS-FHVI dataset, the logarithmic scaling and The best classification effect is achieved, which is much higher than the original ordinal regression method. The best classification effect is achieved, which is 2.65 higher than the original ordinal regression. Figure 5 Shown and The consistency diagram between .

[0139] Table 4 and Consistency between

[0140]

[0141] 4.3 Calculating the visibility label vector of the detection and the true visibility level label vector The consistency of the difference between the part where the elements are 1 and the part where they are 0 (different weight scaling functions)

[0142] For the ordinal consistency calculation method of formula (3), this application only considers (Table 5). On the JS-FHVI dataset, linear scaling achieved the best classification effect, which was much higher than the original ordinal regression method. On the DG-FHVI dataset, logarithmic scaling achieved the best classification effect, which was 1.74 higher than the original ordinal regression. Figure 6 Shown and A consistency graph of the difference between the part where the element is 1 and the part where the element is 0.

[0143] Table 5 and The consistency of the difference between the part where the element is 1 and the part where it is 0

[0144]

[0145] 4.4 Detection Visibility Label Vector and the true visibility level label vector IoU (Intersection over Union) (different weight scaling functions):

[0146] The results of the five scaling functions for the ordinal consistency calculation method of formula (4) are shown in Table 6. On the JS-FHVI dataset, the exponential square root scaling is used. and The best classification results were achieved, which were much higher than the original ordinal regression method. It also achieved the highest classification accuracy, which was 3.97 higher than the original ordinal regression. Figure 7 Shown and IoU graph of .

[0147] Table 6 and IoU

[0148]

[0149] In a feasible implementation, Figure 8 Shown is the comparison of experimental results of the ordinal regression method based on ordinal consistency and the original ordinal regression method at various levels of the JS-FHVI and DG-FHVI datasets. Figure 8 The first row is an example of JS-FHVI, and the second row is an example of DG-FHVI. For examples where the original ordinal regression method detects incorrect visibility levels, the designed method based on ordinal consistency constraints can correct these errors and detect the correct visibility levels.

[0150] In an embodiment of the present invention, a novel visibility detection method based on ordinal consistency constraint is proposed, which constrains the consistency of classifier results by converting the traditional visibility level classification task into a plurality of ordered binary classifiers.

[0151] At the same time, an adaptive ordinal consistency constraint is further proposed, and the consistency is used to weight the training samples, increasing the weight of inconsistent detection samples, so that the model focuses on detecting samples with inconsistent ordinals and improves the accuracy of visibility classification.

[0152] Detailed experiments were conducted on two visibility detection datasets collected from real highway scene datasets. The experimental results show that the visibility detection method based on ordinal consistency constraint proposed in this application can effectively improve the classification accuracy of the visibility detection model.

[0153] Figure 9 1 is a block diagram of a visibility detection device 300 based on ordinal consistency constraint according to an exemplary embodiment. The device 300 is used in a visibility detection method based on ordinal consistency constraint. Figure 9 The device includes an image data acquisition module 310, a model building module 320, a visibility detection module 330, and a detection result calibration module 340.

[0154] An image data acquisition module 310 is configured to acquire image data via a video surveillance device;

[0155] A model building module 320 is used to build a visibility detection model based on ordinal consistency constraints; the visibility detection model includes two parts: an ordinal regression network based on VGG16 and an ordinal consistency constraint loss function;

[0156] The visibility detection module 330 is used to input the image data into the ordinal regression network based on VGG16 to extract the visibility features of the image and obtain the visibility level label classification result;

[0157] The detection result calibration module 340 is used to obtain the visibility level label classification result, assign different weight information to each sample, and calculate the consistency of the intersection and union ratio of the detected visibility label classification result and the true visibility level label based on the ordinal consistency constraint loss function to obtain accurate visibility level classification results and complete visibility detection based on ordinal consistency constraints.

[0158] Optionally, a model building module 320 for a VGG16-based ordinal regression network comprising 13 convolutional network layers, wherein ReLU functions are used after all convolutional layers; max pooling and batch normalization are used after the first, fourth, seventh, tenth, and last convolutional network layers; and a mapping between visibility features and ordinal visibility levels is learned from highway images through VGG16-based ordinal regression.

[0159] The ordinal consistency constraint loss is used to ensure the consistency between the ordinal information learned by the visibility detection model and the true value.

[0160] Optionally, the visibility detection module 330 is configured to input the image data into an ordinal regression network based on VGG16;

[0161] The feature map is reduced to a fixed size through an adaptive average pooling layer of a VGG16-based ordinal regression network;

[0162] Extract deep feature information related to visibility through two fully connected layers;

[0163] Obtain visibility level labels corresponding to the image data, and expand the visibility level labels into vectors that retain the order information of the level labels;

[0164] The ordinal relationship between image visibility features and grade labels is learned through ordinal regression to obtain the visibility grade label classification results.

[0165] Optionally, obtaining a visibility level label corresponding to the image data and expanding the visibility level label into a vector that retains the level label order information includes:

[0166] Get the image , then the corresponding visibility level label is , where W represents the image width, H represents the image height, C represents the number of image channels, i represents the i-th image, and K represents the number of visibility level categories;

[0167] Label the visibility level Encoded as a vector with k elements .

[0168] Optionally, an ordinal relationship between image visibility features and grade labels is learned through ordinal regression to obtain a visibility grade label classification result, including:

[0169] kth element The calculation method is as follows:

[0170] (1)

[0171] in, ;

[0172] use A classifier maps the deep visibility feature embeddings from the two fully connected layers to a binary output visibility label vector consistent with the visibility level label. ;

[0173] calculate The sum of the classifier outputs is used to get the image The final detection visibility level is as follows:

[0174] (2).

[0175] Optionally, the detection result calibration module 340 is used to define the ordinal consistency constraint loss function As shown in the following formula (3):

[0176] (3)

[0177] Where N represents the number of images in the training set. represents the exponential open scaling function, represents the ordinal consistency result, represents the cross entropy loss function;

[0178] Define the exponential scaling function As shown in the following formula (4):

[0179] (4)

[0180] The calculated ordinal consistency results Use the exponential scaling function as the weight value assigned to the sample;

[0181] The cross entropy function is defined as follows:

[0182] (5)

[0183] in, Indicates the number of visibility level categories, represents an indicator function, the result is 1 when the condition is true and 0 when the condition is false; p represents the model prediction probability; Indicates the images; Represents the parameters of the model;

[0184] Get visibility label vector and the true visibility label vector ;

[0185] By calculating the visibility label vector of the detection and the true visibility level label vector The intersection-over-union ratio is used to obtain the ordinal consistency result of the visibility level; and the exponential square root scaling function is used to give higher weights to difficult samples with inconsistent ordinals, so that the model can focus on difficult samples;

[0186] Obtain accurate visibility level classification results and complete visibility detection based on ordinal consistency constraints.

[0187] Optionally, by computing the visibility label vector of the detections and the true visibility level label vector The ordinal consistency results of visibility levels are obtained by using the intersection-union ratio of , including:

[0188] The ordinal consistency of visibility levels is calculated according to the following formula (6):

[0189] (6).

[0190] In an embodiment of the present invention, the present invention proposes a visibility detection method based on ordinal consistency constraints, which constrains the consistency of the classifier results by converting the traditional visibility level classification task into multiple ordered binary classifiers. At the same time, an adaptive ordinal consistency constraint is further proposed, and the consistency is used to weight the training samples, increase the weight of inconsistent detection samples, and improve the accuracy of visibility classification. This method is trained and tested on an image dataset collected by video surveillance equipment on a highway, and the effectiveness of this method is verified through ablation experiments. The experimental results show that the visibility detection method based on ordinal consistency constraints proposed in this application can effectively improve the classification accuracy of the visibility detection model.

[0191] Figure 10 is a structural diagram of a visibility detection device based on ordinal consistency constraint provided by an embodiment of the present invention, such as Figure 10 As shown, the visibility detection device based on ordinal consistency constraint may include the above Figure 9 Optionally, the visibility detection device 410 based on ordinal consistency constraint may include a first processor 2001 .

[0192] Optionally, the visibility detection device 410 based on ordinal consistency constraint may further include a memory 2002 and a transceiver 2003 .

[0193] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0194] The following combination Figure 10 The components of the visibility detection device 410 based on ordinal consistency constraint are described in detail:

[0195] The first processor 2001 is the control center of the visibility detection device 410 based on ordinal consistency constraints, and can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), or application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs) or one or more field programmable gate arrays (FPGAs).

[0196] Optionally, the first processor 2001 may execute various functions of the visibility detection device 410 based on ordinal consistency constraints by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002 .

[0197] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 10 CPU0 and CPU1 are shown in FIG.

[0198] In a specific implementation, as an embodiment, the visibility detection device 410 based on ordinal consistency constraint may also include multiple processors, such as Figure 10 1 and 2. The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). A processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0199] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled by the first processor 2001 for execution. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0200] Alternatively, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, a random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently and be accessed through the interface circuit ( Figure 10 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0201] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0202] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 10 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0203] Optionally, the transceiver 2003 may be integrated with the first processor 2001 or may exist independently and be connected to the visibility detection device 410 through the interface circuit ( Figure 10 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0204] It should be noted that Figure 10 The structure of the visibility detection device 410 based on ordinal consistency constraints shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0205] In addition, the technical effects of the visibility detection device 410 based on ordinal consistency constraints can refer to the technical effects of the visibility detection method based on ordinal consistency constraints described in the above method embodiment, and will not be repeated here.

[0206] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

[0207] It should also be understood that the memory in the embodiments of the present invention may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0208] The above embodiments can be implemented in whole or in part via software, hardware (e.g., circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present invention are fully or partially performed. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable sensor. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired means (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0209] It should be understood that the term "and / or" in this application is merely a description of the relationship between associated objects, indicating that three possible relationships exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this application generally indicates that the associated objects are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0210] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0211] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0212] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0213] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0214] If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for causing a computer device (such as a personal computer, server, or network device) to execute all or part of the steps of the methods described in various embodiments of the present invention.

[0215] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A visibility detection method based on ordinal consistency constraint, characterized in that: The method comprises: S1. Collect image data through video surveillance equipment; S2. Constructing a visibility detection model based on ordinal consistency constraint; the visibility detection model includes two parts: an ordinal regression network based on VGG16 and an ordinal consistency constraint loss function; S3. Inputting the image data into the VGG16-based ordinal regression network to extract the visibility features of the image and obtain a visibility level label classification result; S4. Obtain the visibility level label classification results, assign different weight information to each sample, calculate the consistency of the detected visibility label classification results and the true visibility level label using the intersection-over-union ratio based on the ordinal consistency constraint loss function, and use the exponential square root scaling function to assign higher weights to difficult samples with inconsistent ordinals, so that the model pays more attention to difficult samples during training; Obtain accurate visibility level classification results and complete visibility detection based on ordinal consistency constraints.

2. The visibility detection method based on ordinal consistency constraint according to claim 1, characterized in that S2 include: The VGG16-based ordinal regression network contains 13 convolutional network layers, and the ReLU function is used after all convolutional layers; maximum pooling and batch normalization are used after the first, fourth, seventh, tenth and last convolutional network layers; Learning to extract a mapping of the ordinal relationship between visibility features and visibility levels from highway images through the VGG16-based ordinal regression; The ordinal consistency constraint loss is used to ensure consistency between the ordinal information learned by the visibility detection model and the true value.

3. The visibility detection method based on ordinal consistency constraint according to claim 2, characterized in that: In S3, the image data is input into the VGG16-based ordinal regression network to extract the visibility features of the image and obtain the visibility level label classification result, including: Inputting the image data into the VGG16-based ordinal regression network; The feature map is reduced to a fixed size through the adaptive average pooling layer of the VGG16-based ordinal regression network; Extract deep feature information related to visibility through two fully connected layers; Obtaining a visibility level label corresponding to the image data, and expanding the visibility level label into a vector that retains level label order information; The ordinal relationship between image visibility features and grade labels is learned through ordinal regression to obtain the visibility grade label classification results.

4. The visibility detection method based on ordinal consistency constraint according to claim 3, characterized in that: The obtaining of the visibility level label corresponding to the image data and the expansion of the visibility level label into a vector retaining level label sequence information includes: Get the image The corresponding visibility level label is c i ∈{0, 1, ..., K-1}, where W represents the image width, H represents the image height, C represents the number of image channels, i represents the i-th image, and K represents the number of visibility level categories; Label the visibility level c i Encoded as a vector y with k elements i .

5. The visibility detection method based on ordinal consistency constraint according to claim 4, characterized in that: The ordinal relationship between the image visibility features and the grade labels is learned through ordinal regression to obtain the visibility grade label classification result, including: kth element The calculation method is as follows: Wherein, k=0, 1, ..., K-2; Use K-1 classifiers to map the deep visibility feature embeddings from the two fully connected layers to a binary output visibility label vector consistent with the visibility class label. Calculate the sum of the outputs of K-1 classifiers to get image I i The final detection visibility level is as follows:

6. The visibility detection method based on ordinal consistency constraint according to claim 5, characterized in that: In S4, the visibility level label classification results are obtained, and different weight information is assigned to each sample. The consistency of the detected visibility label classification results and the true visibility level labels is calculated by the intersection-over-union ratio based on the ordinal consistency constraint loss function to obtain accurate visibility level classification results, completing the visibility detection based on the ordinal consistency constraint, including: Define the ordinal consistency constraint loss function L CWCE As shown in the following formula (3): Where N is the number of training set images, σ(·) represents the exponential open scaling function, and CW i Indicates the ordinal consistency result, L CE represents the cross entropy loss function; The exponential root scaling function σ(·) is defined as the following formula (4): The calculated ordinal consistency result CW i Use the exponential scaling function as the weight value assigned to the sample; The cross entropy function is defined as follows: Where K represents the number of visibility level categories, 1(·) represents an indicator function, the result is 1 when the condition is true and 0 when the condition is false; p represents the model prediction probability; I i represents the i-th image; θ represents the parameters of the model; Get visibility label vector and the true visibility label vector y i ; By calculating the visibility label vector of the detection and the true visibility level label vector y i The intersection-over-union ratio is used to obtain the ordinal consistency result of the visibility level; and the exponential square root scaling function is used to give higher weights to difficult samples with inconsistent ordinals, so that the model can focus on difficult samples; Obtain accurate visibility level classification results and complete visibility detection based on ordinal consistency constraints.

7. The visibility detection method based on ordinal consistency constraint according to claim 6, characterized in that: The visibility label vector detected by calculating and the true visibility level label vector y i The ordinal consistency results of visibility levels are obtained by using the intersection-union ratio of , including: The ordinal consistency of visibility levels is calculated according to the following formula (6):

8. A visibility detection device based on ordinal consistency constraints, wherein the visibility detection device based on ordinal consistency constraints is used to implement the visibility detection method based on ordinal consistency constraints according to any one of claims 1 to 7, characterized in that: The device comprises: An image data acquisition module, used to acquire image data through video surveillance equipment; A model building module is used to build a visibility detection model based on ordinal consistency constraints; the visibility detection model includes two parts: an ordinal regression network based on VGG16 and an ordinal consistency constraint loss function; A visibility detection module is used to input the image data into the VGG16-based ordinal regression network to extract the visibility features of the image and obtain a visibility level label classification result; The detection result calibration module is used to obtain the visibility level label classification results, assign different weight information to each sample, calculate the consistency of the detected visibility label classification results and the true visibility level label through the intersection and union ratio based on the ordinal consistency constraint loss function, and use the exponential square root scaling function to assign higher weights to difficult samples with inconsistent ordinals, so that the model can focus on difficult samples during the training process; obtain accurate visibility level classification results and complete visibility detection based on ordinal consistency constraints.

9. A visibility detection device based on ordinal consistency constraints, comprising: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, any one of the visibility detection methods based on ordinal consistency constraints as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the visibility detection methods based on ordinal consistency constraints as described in any one of claims 1 to 7.

Images