An image monitoring bad weather visibility level identification method and system

Abstract

The present application relates to the technical field of image classification, and proposes an image monitoring bad weather visibility grade identification method and system, which divides the to-be-identified image obtained from the monitoring video into multiple sub-images, inputs the multiple sub-images into a trained image classification model for identification and output of a classification result, and determines the visibility grade of the to-be-identified image according to the classification result. Through the identification method of the present application, the intelligent identification of fog and haze weather on the expressway can be completed by using existing facilities, the identification result is based on the area of each monitoring, the accuracy is high, the construction cost and the labor cost are greatly reduced, and the safety problem of the existing fog and haze weather affecting the expressway is well solved.

Description

Technical Field

[0001] This invention relates to the field of image classification technology, specifically to a method and system for identifying visibility levels in adverse weather conditions using image monitoring. Background Technology

[0002] Currently, the total length of expressways in China is increasing year by year. As one of the most important modes of transportation, expressways have made a significant contribution to the country's rapid development. With the construction and operation of expressways, expressway safety has become a top priority. The occurrence of smog reduces visibility on expressways, shortens drivers' field of vision, slows emergency response, and greatly increases the probability of accidents, posing a significant threat to life and property. Therefore, smog is one of the factors that endanger expressway safety.

[0003] Anticipating smog in advance and implementing measures such as traffic control or closure of relevant sections of highways can effectively prevent traffic accidents caused by smog on highways. However, smog forecasts cover a wide area and are not applicable to highways. Currently, there are two main solutions: First, constructing monitoring stations on highways and using specialized equipment to detect visibility at these stations to determine the presence of smog. This method is difficult to implement due to the high cost of construction and the large number and length of highway sections, making dense deployment across all sections impractical. Second, relying on on-site patrols by traffic management personnel to visually monitor the occurrence of smog in real time. This method requires a large workforce and has poor timeliness, often only discovering and reporting smog some time after it has already appeared. Summary of the Invention

[0004] This invention proposes a method and system for identifying visibility levels in adverse weather conditions through image monitoring. It identifies visibility levels in adverse weather conditions by monitoring video data, reducing construction and labor costs, and solving the problems of high cost and poor timeliness in existing highway fog and haze weather monitoring.

[0005] The technical solution of the present invention is as follows:

[0006] A method for identifying visibility levels in adverse weather conditions using image monitoring includes the following steps:

[0007] The image to be identified is obtained from the surveillance video, which comes from the image acquisition device of the area to be monitored;

[0008] The image to be identified is segmented to obtain multiple sub-images;

[0009] The multiple sub-images are input into a trained image classification model to obtain the classification results of the multiple sub-images;

[0010] Based on the classification results of the multiple sub-images, the visibility level of the image to be identified is determined.

[0011] Furthermore, the step of segmenting the image to be identified into multiple sub-images specifically includes:

[0012] The center point of the image to be identified is set as the first-level interest point, and the points are sequentially expanded outward to determine the i-th level interest points, i = 2, 3, ..., n, where the number of i-th level interest points is 2. i ;

[0013] Centered on each point of interest, a region of size W*H is selected as a sub-image and segmented. The number of sub-images is [number missing]. The sub-image containing the i-th level interest point is called the i-th level sub-image.

[0014] Furthermore, the classification results of the sub-images include: clear, moderate, and severe;

[0015] The step of determining the visibility level of the image to be identified based on the classification results of the multiple sub-images specifically includes:

[0016] Based on the classification result of the sub-image, a preset assignment function is called to assign a visibility value to the sub-image;

[0017] Based on the visibility assignments of the multiple sub-images, the visibility value of the image to be identified is calculated;

[0018] Based on the visibility value of the image to be identified, a preset classification function is invoked to determine the visibility level of the image to be identified.

[0019] Furthermore, each of the sub-images is pre-assigned a weight according to its level. The step of calculating the visibility value of the image to be identified based on the visibility assignments of multiple sub-images specifically includes:

[0020] The visibility values ​​of the multiple sub-images are summed according to their weights to obtain the visibility value of the image to be identified.

[0021] Furthermore, the training method for the image classification model includes,

[0022] Acquire a large number of image samples under adverse weather conditions. Define the center point of each image sample as a first-level point of interest (POI), and then expand outwards sequentially, defining the POIs as the i-th level, where i = 2, 3, ..., n, and the number of i-th level POIs is 2^n. i ;

[0023] Centered on each point of interest, a region of size W*H is selected as the image region of interest. The number of image regions of interest for each image sample is [number missing].

[0024] Extract the regions of interest from all the image samples and store them as new image samples;

[0025] The new image samples are classified and labeled into three categories: clear, moderate, and severe, and then input into the image classification model for training.

[0026] A system for identifying visibility levels in adverse weather conditions using image monitoring, comprising:

[0027] The first obtaining unit is used to obtain an image to be identified from the surveillance video, wherein the surveillance video comes from an image acquisition device of the area to be monitored;

[0028] The first segmentation unit is used to segment the image to be identified to obtain multiple sub-images;

[0029] The first input unit is used to input the multiple sub-images into the trained image classification model to obtain the classification results of the multiple sub-images;

[0030] The first processing unit is used to determine the visibility level of the image to be identified based on the classification results of the multiple sub-images.

[0031] Furthermore, it also includes:

[0032] The second processing unit is used to set the center point of the image to be identified as the first-level interest point, and to sequentially expand outward to determine the i-th level interest point, i = 2, 3, ..., n, where the number of i-th level interest points is 2i;

[0033] The third processing unit is used to select a region of size W*H pixels centered on each point of interest as a sub-image for segmentation, and the number of sub-images is [number missing]. The sub-image containing the i-th level interest point is called the i-th level sub-image.

[0034] Furthermore, it also includes:

[0035] The first assignment unit is used to call a preset assignment function to assign a value to the visibility of the sub-image based on the classification result of the sub-image;

[0036] The first calculation unit is used to calculate the visibility value of the image to be identified based on the visibility assignments of the multiple sub-images.

[0037] The fourth processing unit is used to call a preset classification function based on the visibility value of the image to be identified to determine the visibility level of the image to be identified.

[0038] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the visibility level identification method.

[0039] The working principle and beneficial effects of this invention are as follows:

[0040] 1. This invention first constructs an image classification model, inputs video surveillance images into the model, and obtains the visibility level of the video area. Through the identification method of this invention, intelligent identification of haze on highways can be achieved using existing facilities. The identification results are based on the location of each monitoring area, with high accuracy, greatly reducing construction and labor costs, and effectively solving the existing safety problem of haze affecting highways.

[0041] 2. This invention improves the accuracy of visibility level recognition based on images by segmenting surveillance images based on points of interest and determining the visibility level of the image to be identified based on the classification results of multiple sub-images. Without segmentation, the scene captured in the entire image is large and involves complex environmental areas. Classifying the entire image as data results in a large amount of feature interference, making it impossible to effectively learn the visibility level category information. Furthermore, image segmentation increases the training samples for the image classification model, leading to better training results.

[0042] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0043] Figure 1 This is a flowchart of the present invention;

[0044] Figure 2 This is a schematic diagram illustrating the hierarchical classification of points of interest in this invention;

[0045] Figure 3 This is a schematic diagram of the Resnet network structure in this invention. Detailed Implementation

[0046] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0047] Example 1

[0048] This embodiment proposes a method for identifying visibility levels in adverse weather conditions using image monitoring. It identifies visibility levels based on highway surveillance video data, thereby determining the severity of smog. Specifically, it includes:

[0049] (1) Extraction of image interest points and regions of interest

[0050] Acquire a large number of image samples under adverse weather conditions. Within each image, define points of interest (POIs) by expanding outwards from the image center. Furthermore, define the levels of each POI from the image center outwards. Figure 2 As shown,

[0051] Level 1 point of interest: 1 (image center point);

[0052] Level 2 points of interest: 2 2 (Select one circle outward from the center point);

[0053] Level 3 Interest Points: 2 3 (Select one circle outward from the second level of interest);

[0054] Level 4 Interest Points: 2 4 (Select one circle outward from the 3rd level point of interest);

[0055] By cropping the image region of interest from four levels of interest points, 29 regions of interest can be obtained.

[0056] At each point of interest, an image region of size W*H is selected centered on the point of interest, and the image region is extracted as a new image sample.

[0057] Because performing visibility classification and recognition on the entire image is ineffective, analysis reveals that the scene captured in the image is large and involves a complex environment. Classifying the entire image as data results in significant feature interference, and with a small sample size, the model cannot effectively learn visibility level category information. Therefore, segmenting the image based on points of interest not only increases the sample size but also avoids feature interference problems caused by complex environmental regions.

[0058] (2) Class sample production of the new algorithm

[0059] All collected images were analyzed according to the method in step (1) to determine the points of interest, and all regions of interest were extracted and stored as new image samples. Then, three categories (clear, medium, and heavy) were defined to represent the visibility of each new image sample, and all samples were manually classified and labeled according to these three categories. All samples were made into three-category (clear, medium, and heavy) sample sets, with clear labeled as 1, medium labeled as 2, and heavy labeled as 3, thus completing the production of small image category sample data.

[0060] (3) Image classification model training

[0061] An image classification model is built based on the ResNet-50 deep convolutional neural network. Then, the three-class sample sets prepared in step (2) are used to train the classification model. The sample data set includes images and label information. According to the input requirements of the ResNet model, all images need to be converted to a resolution of 3*224*224 pixels first. Then, these sample datasets are input into the built ResNet-50 model for training. The initial learning rate is set to 0.001. The training process adopts a momentum-based optimization algorithm. The training of the model can be completed by setting the number of iterations to 100,000.

[0062] The principle behind the ResNet deep learning classification model is as follows:

[0063] ResNet is a deep neural network with a convolutional structure. Convolutional structures can reduce the amount of memory occupied by deep networks. Its three key operations are local receptive fields, weight sharing, and pooling layers, which effectively reduce the number of network parameters and alleviate the overfitting problem of the model.

[0064] Its common network structure forms are as follows Figure 3 As shown in the diagram, the five basic building blocks of a convolutional neural network are: input layer, convolutional layer, pooling layer, fully connected layer, and output layer.

[0065] Input formula: V = conv2(W, X,"valid") + b, Output formula:

[0066] The input-output formulas above apply to each convolutional layer. Each convolutional layer has a different weight matrix W, and W, X, and Y are in matrix form. For the last fully connected layer, let's call it the L-th layer, the output is a vector yL, and the expected output is d. Then we have the total error formula.

[0067] `Conv2()` is a function for convolution operations. The third parameter, `valid`, specifies the type of convolution operation; the preceding convolution mode is of type `valid`. `W` is the convolution kernel matrix, `X` is the input matrix, and `b` is the bias. It is the activation function. In the total error, d and y are the vectors of the expected output and the network output, respectively.

[0068] ResNet is trained using gradient descent and backpropagation algorithms, so the gradient formula for the fully connected layers is exactly the same as that for the backpropagation network. Below are the convolution formulas for the convolutional and pooling layers:

[0069] The most important part of ResNet is the residual learning unit.

[0070] For a stacked layer structure (composed of several layers), when the input is x, the learned features are denoted as H(x). Now we want it to learn the residual F(x) = H(x) - x, so the original learned features are actually F(x) + x. When the residual is 0, the stacked layer has only performed an identity mapping, and at least the network performance will not decrease. In fact, the residual will not be 0, which will also allow the stacked layer to learn new features based on the input features, thus achieving better performance.

[0071] (4) Category identification of images of unknown point of interest regions

[0072] For an image that needs visibility level recognition, determine the image interest points according to the method in step (1), and extract all the image interest regions to obtain 29 small images of unknown categories (i.e., sub-images). Then, use the image classification model trained in step (3) to predict the category of each of the 29 small images to obtain their respective categories.

[0073] (5) Visibility score of the region of interest in the image

[0074] A score of 1 is awarded for small images predicted by the model to be of the "clear" category; a score of 0.5 is awarded for small images predicted by the model to be of the "medium" category; and a score of 0 is awarded for small images predicted to be of the "heavy" category. Then, the scores of the 29 small images representing regions of interest extracted from each image are recorded.

[0075] (6) Setting the weight coefficients of image interest regions

[0076] Based on the characteristics of image interest points, different scoring weight coefficients are assigned to the small images of image interest regions extracted from interest points of different levels. The weight coefficients are as follows:

[0077] Weight coefficient of interest region extracted from Level 1 interest points: w1

[0078] Weight coefficient of interest region extracted from Level 2 interest points: w2

[0079] Weight coefficient of interest region extracted from Level 3 interest points: w3

[0080] Weight coefficient of interest region extracted from Level 4 interest points: w4

[0081] (7) Calculate the visibility score of the entire image.

[0082] Score of the whole image Among them, S ij W represents the score of the interest region in the j-th image within the i-th level interest point. i This represents the weight of the i-th level interest point.

[0083] (8) Visibility level decision for the entire image

[0084] Rules for fog and haze visibility levels have been formulated. Based on the requirements of the national standard GB / T 31445-2015 "Traffic Safety Control Conditions on Highways in Fog" and the "Meteorological Industry Standard of the People's Republic of China", visibility levels are divided into six levels after comprehensive consideration, as shown in the table below:

[0085] Table 1. Highway Haze Visibility Levels

[0086]

[0087]

[0088] Based on the established six visibility levels for highways, a threshold range was set for each level. The threshold ranges for the six levels were determined through repeated experiments, as shown in the table below:

[0089] Table 2 Threshold ranges for six levels

[0090] Visibility level Threshold range Level 6 Th5≤score≤Th6 Level 5 Th4≤score<Th5 Level 4 Th3≤score<Th4 Level 3 Th2≤score<Th3 Level 2 Th1≤score<Th2 Level 1 0≤score<Th1

[0091] Using the threshold regions in the table above, a decision is made on the score of the entire image calculated in step 7). Images that fall within the corresponding threshold range have the corresponding haze visibility level.

[0092] (9) Detection of adverse weather conditions and alarm for low visibility

[0093] The visibility level decision in step (8) determines the severity level of the current monitoring point. For severe conditions that meet Level 1 or Level 2 criteria, the monitoring point is identified as an adverse meteorological environment, and a low visibility alarm is triggered. The alarm information can be sent to the front end for display via an interface.

[0094] Example 2

[0095] This embodiment proposes an image monitoring system for identifying visibility levels in adverse weather conditions, including:

[0096] The first obtaining unit is used to obtain an image to be identified from the surveillance video, wherein the surveillance video comes from an image acquisition device of the area to be monitored;

[0097] The first segmentation unit is used to segment the image to be identified to obtain multiple sub-images;

[0098] The first input unit is used to input the multiple sub-images into the trained image classification model to obtain the classification results of the multiple sub-images;

[0099] The first processing unit is used to determine the visibility level of the image to be identified based on the classification results of the multiple sub-images.

[0100] Furthermore, this embodiment also includes:

[0101] The second processing unit is used to set the center point of the image to be identified as the first-level interest point, and to sequentially expand outward to determine the i-th level interest point, i = 2, 3, ..., n, where the number of i-th level interest points is 2i;

[0102] The third processing unit is used to select a region of size W*H as a sub-image centered on each point of interest, and to segment it. The number of sub-images is , where the sub-image containing the i-th level point of interest is the i-th level sub-image.

[0103] Furthermore, this embodiment also includes:

[0104] The first assignment unit is used to call a preset assignment function to assign a value to the visibility of the sub-image based on the classification result of the sub-image;

[0105] The first calculation unit is used to calculate the visibility value of the image to be identified based on the visibility assignments of the multiple sub-images.

[0106] The fourth processing unit is used to call a preset classification function based on the visibility value of the image to be identified to determine the visibility level of the image to be identified.

[0107] Example 3

[0108] This embodiment proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the visibility level identification method.

[0109] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying visibility levels in adverse weather conditions using image monitoring, characterized in that, Includes the following steps: The image to be identified is obtained from the surveillance video, which comes from the image acquisition device of the area to be monitored; The image to be identified is segmented to obtain multiple sub-images; The multiple sub-images are input into a trained image classification model to obtain the classification results of the multiple sub-images; Based on the classification results of the multiple sub-images, the visibility level of the image to be identified is determined; The step of segmenting the image to be identified to obtain multiple sub-images specifically includes: The center point of the image to be identified is set as the first level interest point, and is diffused outward in turn to determine the i-level interest point, i=2, 3, …, n, wherein the number of the i-level interest points is 2 i ; Centered on each point of interest, select a pixel size of The region is used as a sub-image for segmentation, and the number of sub-images is [number missing]. , where the sub-image containing the i-th level interest point is called the i-th level sub-image.

2. The method for identifying visibility levels in adverse weather conditions using image monitoring according to claim 1, characterized in that, The classification results of the sub-images include: clear, moderate, and severe; The step of determining the visibility level of the image to be identified based on the classification results of the multiple sub-images specifically includes: Based on the classification result of the sub-image, a preset assignment function is called to assign a visibility value to the sub-image; Based on the visibility assignments of the multiple sub-images, the visibility value of the image to be identified is calculated; Based on the visibility value of the image to be identified, a preset classification function is invoked to determine the visibility level of the image to be identified.

3. The method for identifying visibility levels in adverse weather conditions using image monitoring according to claim 2, characterized in that, Each of the sub-images is pre-assigned a weight according to a level. The step of calculating the visibility value of the image to be identified based on the visibility assignments of the multiple sub-images specifically includes: The visibility values ​​of the multiple sub-images are summed according to their weights to obtain the visibility value of the image to be identified.

4. The method for identifying visibility levels in adverse weather conditions using image monitoring according to claim 1, characterized in that, The training method for the image classification model includes, Acquire a large number of image samples under adverse weather conditions. Define the center point of each image sample as a first-level point of interest (POI), and then expand outwards sequentially, defining the POIs as the i-th level, i = 2, 3, ..., n, where the number of i-th level POIs is 2^n. i ; Centered on each point of interest, select a pixel size of The regions are designated as regions of interest (ROIs) for each image sample, and the number of ROIs for each image sample is [number missing]. ; Extract the regions of interest from all the image samples and store them as new image samples; The new image samples are classified and labeled into three categories: clear, moderate, and severe, and then input into the image classification model for training.

5. A system for identifying visibility levels in adverse weather conditions using image monitoring, characterized in that, include: The first obtaining unit is used to obtain an image to be identified from the surveillance video, wherein the surveillance video comes from an image acquisition device of the area to be monitored; The first segmentation unit is used to segment the image to be identified to obtain multiple sub-images; The first input unit is used to input the multiple sub-images into the trained image classification model to obtain the classification results of the multiple sub-images; The first processing unit is used to determine the visibility level of the image to be identified based on the classification results of the multiple sub-images; Also includes: The second processing unit is used to set the center point of the image to be identified as the first-level interest point, and to sequentially expand outward to determine the i-th level interest point, i=2,3,...,n, where the number of i-th level interest points is 2i; The third processing unit is used to select a pixel size centered on each point of interest. The region is used as a sub-image for segmentation, and the number of sub-images is [number missing]. , where the sub-image containing the i-th level interest point is called the i-th level sub-image.

6. The image monitoring system for identifying visibility levels in adverse weather conditions according to claim 5, characterized in that, Also includes: The first assignment unit is used to call a preset assignment function to assign a value to the visibility of the sub-image based on the classification result of the sub-image; The first calculation unit is used to calculate the visibility value of the image to be identified based on the visibility assignments of the multiple sub-images. The fourth processing unit is used to call a preset classification function based on the visibility value of the image to be identified to determine the visibility level of the image to be identified.

7. A computer-readable storage medium, wherein, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the visibility level identification method as described in any one of claims 1-4.

Images