A monitoring operation monitoring method based on image quality

Abstract

The present invention discloses a monitoring operation monitoring method based on image quality, comprising the following steps: collecting monitoring images; conducting quality evaluation on the collected monitoring images by constructing a good image quality score evaluation model to obtain a quality evaluation image quality score; conducting operation status evaluation on the collected monitoring images according to the image quality score; dynamically displaying the changes in the operation status of the monitoring images in real time; and monitoring the monitoring operation according to the operation status of the monitoring images. The present invention realizes real-time online detection, and can objectively, quickly, efficiently, and digitally evaluate the quality of monitoring images / videos, facilitate relevant personnel to conduct inspections and repairs, improve the efficiency of abnormal monitoring discovery, and save a lot of human resource costs.

Description

Technical Field

[0001] The invention belongs to the field of image processing, and in particular relates to a monitoring operation monitoring method based on image quality. Background Art

[0002] With the rapid development of monitoring system technology and the rapid reduction of application costs, and in order to meet more security needs, more and more monitoring systems are used in all aspects of people's lives. With the promotion of monitoring systems, some potential problems have also emerged, among which the uneven quality of monitoring video images is particularly obvious. In monitoring systems, problems such as out-of-focus blur and block effects often occur, which seriously affect the image quality. If they are not handled in time, the monitoring effect will be affected. Today's monitoring systems are very large, with hundreds or thousands of monitoring videos. In this case, it is unrealistic to employ a large number of people to subjectively evaluate the image quality of a large number of monitoring videos continuously. Therefore, how to evaluate the quality of monitoring video images in real time and objectively, so that the video quality of each subnet meets the needs of monitoring purposes, has become a new direction in the field of monitoring video research.

[0003] On the other hand, the evaluation standards of surveillance images and the supervision of surveillance systems also have a great impact on the application and expansion of surveillance systems. Traditional surveillance image evaluation uses subjective evaluation methods, which is time-consuming and labor-intensive, and has many inconveniences in practice. At the same time, the supervision of surveillance systems focuses on the management of engineering construction, while ignoring the long-term supervision of the system after delivery. Whether the images of surveillance equipment are still clear and usable after a period of operation cannot be effectively guaranteed. Summary of the invention

[0004] Purpose of the invention: In order to overcome the deficiencies in the prior art, a monitoring operation monitoring method based on image quality is provided, which realizes real-time online detection and can objectively, quickly, efficiently and digitally evaluate the monitoring image / video quality, facilitate relevant personnel to conduct inspections and repairs, improve the efficiency of monitoring anomaly detection, and save a lot of human resource costs.

[0005] Technical solution: To achieve the above purpose, the present invention provides a monitoring operation monitoring method based on image quality, comprising the following steps:

[0006] S1: Collect surveillance images;

[0007] S2: Perform quality evaluation on the collected monitoring images by constructing the image quality score evaluation model to obtain the quality evaluation image quality score;

[0008] S3: Evaluate the operating status of the collected monitoring images according to the image quality score;

[0009] S4: Dynamically display the changes in the running status of the monitoring images in real time;

[0010] S5: Monitor the monitoring operation according to the running status of the monitoring images.

[0011] Furthermore, in the step S1, the real-time acquisition and transmission of the monitoring images are realized through the PC host, the Raspberry Pi main board and the USB camera, which specifically includes the following:

[0012] A1: Burn the MATLAB image on the SD card: The USB camera is connected to the Raspberry Pi main board. The Raspberry Pi main board is operated through MATLAB on the PC host to control the camera to acquire video images in real time. First, insert the SD card into the host card slot, open the hardware resource package of MATLAB, select the Raspberry Pi hardware support package of MATLAB Simulink for download and installation, burn the MATLAB image into the SD card, select the connection method between MATLAB and the Raspberry Pi main board as WIFI connection, input the hotspot ID and password, and the MATLAB image burning of the wireless connection can be completed according to the steps;

[0013] A2: Connect MATLAB, the Raspberry Pi main board and the USB camera: Connect the USB camera to the USB2.0 interface of the Raspberry Pi main board, insert the burned SD card into the card slot of the Raspberry Pi main board and power on, turn on the WIFI hotspot to make the Raspberry Pi main board and the host connected to the same network, and input re = raspi in the MATLAB command line to view the attributes of the Raspberry Pi main board;

[0014] A3: Control the Raspberry Pi main board to acquire images from the USB camera in real time through MATLAB: Open the Simulink module of MATLAB, create a Simulink file, find the support package of Simulink for the Raspberry Pi in the resource library, add the required components (camera, SDL display, etc.) to build a model, wire the components, and then the real-time acquisition of video images can start.

[0015] Furthermore, the construction method of the image quality score evaluation model in the step S2 is as follows:

[0016] B1: Extract 5 image quality-related features, namely histogram of oriented gradients feature, local gradient intensity histogram feature, local brightness histogram feature, local gradient intensity standard deviation histogram feature and local brightness standard deviation histogram feature;

[0017] B2: Calculate 5 feature maps of the reference image and the distorted image respectively based on the extracted features, and calculate the difference between the feature maps of the reference image and the distorted image to obtain the difference feature map;

[0018] B3: The mean of the difference feature map is used to represent the different characteristic differences between the reference image and the distorted image, and the difference values ​​corresponding to the five feature maps of each distorted image are fused to obtain the difference vector;

[0019] B4: The image quality score evaluation model is obtained by learning the mapping relationship between the difference vector and the image quality score through SVR.

[0020] Furthermore, the method for extracting the five image quality related features in step B1 is:

[0021] The steps of directional gradient histogram feature extraction are as follows:

[0022] C1: Use formula (1) to perform gamma correction on the image;

[0023] In the formula, R(i,j), G(i,j) and B(i,j) correspond to the RGB color channel values ​​of the image at (i,j), and I(i,j) is the grayscale value of the pixel at (i,j) after correction.

[0024]

[0025] C2: Use formula (2) and formula (3) to calculate the horizontal gradient and vertical gradient strength of the image at (i, j) respectively;

[0026] In the formula, Gx(i,j) and Gy(i,j) correspond to the horizontal gradient and vertical gradient at (i,j) respectively.

[0027] G x (i,j)=I(i+1,j)-I(i-1,j) (2)

[0028] G y (i,j)=I(i,j+1)-I(i,j-1) (3)

[0029] C3: Calculate the gradient intensity value G(i,j) and the gradient direction θ of the image at (i,j) using the horizontal gradient and vertical gradient obtained in step C2. The specific calculation method is shown in formula (4) and formula (5). Since the gradient direction will be divided into quadrants according to the positive and negative angles of Gx(i,j) and θ, the range of θ is [0,360°];

[0030]

[0031] C4: Divide the image into non-overlapping connected areas of the same size and define them as cells. Divide the image into cells of the same size and use the weighted method to calculate the gradient direction histogram of each cell. The gradient direction of each cell is evenly divided into 9 histogram channels. The weight is the gradient intensity corresponding to the gradient direction, and the directional gradient histogram is obtained.

[0032] C5: The histogram of each cell is normalized using formula (6), where hist i is the i-th value in the histogram vector corresponding to a single cell, HIST i is the i-th value in the histogram vector corresponding to the normalized cell; define a 2*2 cell area as a block, connect the histograms of the 4 cells in each block to obtain a feature vector with a length of 36, and define the directional gradient histogram after block normalization as H 1 ;

[0033]

[0034] The local gradient intensity histogram feature extraction process is as follows:

[0035] D1: gamma correction of the image is performed using formula (1);

[0036] D2: Use formula (2) and formula (3) to calculate the gradient strength of each pixel in the image in the horizontal and vertical directions respectively, and use formula (4) to obtain the gradient strength feature map of the image;

[0037] D3: Use formula (7) to normalize the local gradient intensity feature map of the image;

[0038] In the formula, G(i,j) represents the gradient strength at (i,j), G min and G max Represents the maximum and minimum values ​​of the local gradient intensity feature map of the reference image and the distorted image, G n (i,j) represents the value at (i,j) after normalization;

[0039]

[0040] D4: Divide the image into cells of the same size, divide the gradient intensity of each cell into 10 histogram channels and calculate the gradient intensity histogram H of each cell 2 ;

[0041] The extraction process of local brightness histogram features is as follows:

[0042] E1: Normalize the brightness of the reference image and the distorted image;

[0043] The normalization formula is shown in (8), where I(i,j) represents the pixel value at (i,j), I min and I max Indicates the maximum and minimum brightness of the reference image and the distorted image, I n (i,j) represents the value at (i,j) after normalization;

[0044]

[0045] E2: Divide the image into cells of the same size, divide the brightness of each cell into 10 histogram channels, and obtain the local brightness histogram H 3 ;

[0046] The calculation process of the local gradient intensity standard deviation is as follows:

[0047] F1: gamma correction of the image using formula (1);

[0048] F2: Use formula (2) and formula (3) to calculate the gradient strength of each pixel in the image in the horizontal and vertical directions respectively, and use formula (4) to obtain the gradient strength feature map of the image, and use formula (8) to realize the normalization of the local gradient strength feature map;

[0049] F3: Divide the image into cells of the same size, calculate the standard deviation of each cell, and obtain the local gradient intensity standard deviation feature map;

[0050] F4: Divide the local gradient intensity standard deviation feature map into blocks of the same size. The size of each block is set to 6*6 cells. Divide the gradient intensity standard deviation of each block into 10 histogram channels on average. Statistically obtain the local gradient intensity standard deviation histogram H 4 .

[0051] The local brightness standard deviation reflects the degree of brightness fluctuation in the local range and can effectively measure the impact of contrast distortion on image quality. The extraction process of the local brightness standard deviation histogram feature is as follows:

[0052] G1: Use formula (8) to normalize the image;

[0053] G2: Divide the image into cells of the same size, calculate the standard deviation of each cell, and obtain the local brightness standard deviation feature map;

[0054] G3: Divide the local brightness standard deviation feature map into blocks of the same size, set the block size to 6*6 cells, and evenly divide the brightness standard deviation of each block into 10 histogram channels to obtain the local brightness standard deviation histogram H 5 .

[0055] Furthermore, the operation status evaluation in step S3 includes image quality score and monitoring equipment quality, wherein the monitoring equipment quality is obtained by distinguishing the image distortion type.

[0056] Furthermore, the specific method for determining the type of image distortion in step S3 is:

[0057] The BRISQUE natural image feature algorithm is selected. First, the block MSCN feature matrix of the image is calculated, the block MSCN distribution is calculated, and two parameters are obtained by fitting with a symmetric Gaussian curve as two eigenvalues. Then the matrix of the inner product of the image block MSCN in four directions is calculated, and the distribution is fitted with an asymmetric Gaussian curve to obtain four parameters, a total of 16 features are obtained. Finally, the image is reduced by half, and the first two processes are repeated to obtain a total of 36 eigenvalues, which are the BRISQUE image features. Five high-quality surveillance images of different scenes and different times are taken. A distortion simulation algorithm is used. For each high-quality surveillance image, 500 distorted surveillance images with 5 distortion types and 10 distortion levels are generated as the distortion surveillance image database and the distortion type labels of the human eye calibration data set images are calculated. The BRISQUE features of each image are calculated. For each feature, SVM is used to train a classifier that maps the image feature to whether the distortion type exists. This process is repeated for the five distortion types studied to obtain classifiers for whether the five distortion types exist or not. The image distortion type is distinguished by the classifier.

[0058] Furthermore, the change of the image operation status in step S4 includes the change of the image quality score and the quality of the monitoring device, wherein the display method of the change of the image quality score is:

[0059] Several quality scores are read from the database and displayed on the interface in the form of a line graph, with the horizontal axis being time and the vertical axis being the image quality score;

[0060] The quality status of the monitoring device is displayed as follows: for images with a quality score higher than a threshold, it is judged that there is no problem, and the monitoring device displays it as excellent; for images with a quality score lower than a threshold, it is judged that there is a problem, and the specific image distortion type is displayed.

[0061] The present invention studies an objective evaluation method of image quality suitable for video surveillance system applications, and designs and implements an image quality evaluation system that can be used in video surveillance systems to improve evaluation efficiency, thereby providing effective technical support for the inspection and acceptance of engineering construction and the long-term supervision of system operation.

[0062] The present invention is based on an image quality evaluation method, and realizes the function of predicting the image quality of the image collected by the monitoring camera through the quality evaluation method, and displaying the quality score in the form of dynamic coordinates on the software front-end interface. The present invention proposes for the first time a full-reference image quality evaluation method for atomized images, which can accurately analyze the quality of monitoring images under the influence of atomization, and then reflect the operating status of the monitoring equipment, and has the advantages of fast speed and high recognition accuracy. The present invention runs the image quality evaluation algorithm through the Raspberry Pi motherboard to monitor the quality of the image collected by the camera in real time, displays the monitoring image and its quality analysis results through the PC front-end interface, and finally realizes real-time monitoring of the operating status of the monitoring equipment.

[0063] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0064] 1. Compared with traditional manual experience appraisal and monitoring system behavior recognition, the present invention can judge the quality of monitoring images / videos in real time, quickly and non-destructively, making it convenient for relevant personnel to conduct inspections and repairs, improving the efficiency of monitoring anomaly detection and saving a lot of human resource costs.

[0065] 2. The present invention provides a new quality detection method for monitoring image / video quality control, provides a basis for quality supervision of market monitoring images / videos, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 This is a flow chart of monitoring video image processing in the present invention;

[0067] Figure 2 Schematic diagram of the instruments (Raspberry Pi motherboard, USB camera) used in this embodiment;

[0068] Figure 3 is the monitoring image collected in this embodiment;

[0069] Figure 4 is a flow chart of the image quality evaluation algorithm in this embodiment;

[0070] Figure 5 This is an interface display diagram of the monitoring image quality assessment system in this embodiment;

[0071] Figure 6 This is a diagram showing the results of the monitoring image quality assessment system in this embodiment. DETAILED DESCRIPTION

[0072] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0073] The present invention provides a monitoring operation monitoring system based on image quality, referring to Figure 2 In this embodiment, the monitoring system uses the following instruments: Raspberry Pi 4th generation B model, 8GB, equipped with two USB 2.0 interfaces and two USB 3.0 interfaces; HIKVISION USB camera; Kingston memory card; Raspberry Pi motherboard.

[0074] The connection of the monitoring system in this embodiment is: connect the USB camera to the USB 2.0 interface of Raspberry Pi 4th generation B type, insert the SD card with the MATLAB image burned into the card slot of the Raspberry Pi mainboard, charge and start the Raspberry Pi mainboard, turn on the wireless network, and connect the USB camera, Raspberry Pi and MATLAB on the computer.

[0075] Based on the above monitoring system, this embodiment provides a monitoring operation monitoring method based on image quality, such as Figure 1 As shown, it includes the following steps:

[0076] S1: Collect monitoring images:

[0077] Place the connected USB camera and the powered-on Raspberry Pi in a fixed location, perform simulation through the SIMULINK module of MATLAB on the computer, and transmit the updated video images frame by frame to the SIMULINK display screen in real time;

[0078] The monitoring images collected in this embodiment are as follows: Figure 3 shown.

[0079] S2: Perform quality evaluation on the collected monitoring images by constructing the image quality score evaluation model to obtain the quality evaluation image quality score;

[0080] S3: Evaluate the operating status of the collected monitoring images according to the image quality score;

[0081] S4: Real-time dynamic display of changes in the operating status of monitoring images;

[0082] S5: Monitor the monitoring operation status according to the monitoring image operation status.

[0083] Reference Figure 4In this embodiment, the method for constructing the image quality score evaluation model in step S2 is:

[0084] B1: Extract five image quality related features, namely, oriented gradient histogram feature, local gradient intensity histogram feature, local brightness histogram feature, local gradient intensity standard deviation histogram feature and local brightness standard deviation histogram feature;

[0085] B2: Calculate five feature maps of the reference image and the distorted image based on the extracted features, and calculate the difference between the feature maps of the reference image and the distorted image to obtain a difference feature map;

[0086] B3: The mean of the difference feature map is used to represent the different characteristic differences between the reference image and the distorted image, and the difference values ​​corresponding to the five feature maps of each distorted image are fused to obtain the difference vector;

[0087] B4: The image quality score evaluation model is obtained by learning the mapping relationship between the difference vector and the image quality score through SVR.

[0088] The method for extracting the five image quality related features in step B1 of this embodiment is:

[0089] The steps of directional gradient histogram feature extraction are as follows:

[0090] C1: Use formula (1) to perform gamma correction on the image;

[0091] In the formula, R(i,j), G(i,j) and B(i,j) correspond to the RGB color channel values ​​of the image at (i,j), and I(i,j) is the grayscale value of the pixel at (i,j) after correction.

[0092]

[0093] C2: Use formula (2) and formula (3) to calculate the horizontal gradient and vertical gradient strength of the image at (i, j) respectively;

[0094] In the formula, Gx(i,j) and Gy(i,j) correspond to the horizontal gradient and vertical gradient at (i,j) respectively.

[0095] G x (i,j)=I(i+1,j)-I(i-1,j) (2)

[0096] G y (i,j)=I(i,j+1)-I(i,j-1) (3)

[0097] C3: Calculate the gradient intensity value G(i,j) and the gradient direction θ of the image at (i,j) using the horizontal gradient and vertical gradient obtained in step C2. The specific calculation method is shown in formula (4) and formula (5). Since the gradient direction will be divided into quadrants according to the positive and negative angles of Gx(i,j) and θ, the range of θ is [0,360°];

[0098]

[0099] C4: Divide the image into non-overlapping connected areas of the same size and define them as cells. Divide the image into cells of the same size and use the weighted method to calculate the gradient direction histogram of each cell. The gradient direction of each cell is evenly divided into 9 histogram channels. The weight is the gradient intensity corresponding to the gradient direction, and the directional gradient histogram is obtained.

[0100] C5: The histogram of each cell is normalized using formula (6), where hist i is the i-th value in the histogram vector corresponding to a single cell, HIST i is the i-th value in the histogram vector corresponding to the normalized cell; define a 2*2 cell area as a block, connect the histograms of the 4 cells in each block to obtain a feature vector with a length of 36, and define the directional gradient histogram after block normalization as H 1 ;

[0101]

[0102] The local gradient intensity histogram feature extraction process is as follows:

[0103] D1: gamma correction of the image is performed using formula (1);

[0104] D2: Use formula (2) and formula (3) to calculate the gradient strength of each pixel in the image in the horizontal and vertical directions respectively, and use formula (4) to obtain the gradient strength feature map of the image;

[0105] D3: Use formula (7) to normalize the local gradient intensity feature map of the image;

[0106] In the formula, G(i,j) represents the gradient strength at (i,j), G min and G max Represents the maximum and minimum values ​​of the local gradient intensity feature map of the reference image and the distorted image, G n (i,j) represents the value at (i,j) after normalization;

[0107]

[0108] D4: Divide the image into cells of the same size, divide the gradient intensity of each cell into 10 histogram channels and calculate the gradient intensity histogram H of each cell 2 ;

[0109] The extraction process of local brightness histogram features is as follows:

[0110] E1: Normalize the brightness of the reference image and the distorted image;

[0111] The normalization formula is shown in (8), where I(i,j) represents the pixel value at (i,j), I min and I max Indicates the maximum and minimum brightness of the reference image and the distorted image, I n (i,j) represents the value at (i,j) after normalization;

[0112]

[0113] E2: Divide the image into cells of the same size, divide the brightness of each cell into 10 histogram channels, and obtain the local brightness histogram H 3 ;

[0114] The calculation process of the local gradient intensity standard deviation is as follows:

[0115] F1: gamma correction of the image using formula (1);

[0116] F2: Use formula (2) and formula (3) to calculate the gradient strength of each pixel in the image in the horizontal and vertical directions respectively, and use formula (4) to obtain the gradient strength feature map of the image, and use formula (8) to realize the normalization of the local gradient strength feature map;

[0117] F3: Divide the image into cells of the same size, calculate the standard deviation of each cell, and obtain the local gradient intensity standard deviation feature map;

[0118] F4: Divide the local gradient intensity standard deviation feature map into blocks of the same size. The size of each block is set to 6*6 cells. Divide the gradient intensity standard deviation of each block into 10 histogram channels on average. Statistically obtain the local gradient intensity standard deviation histogram H 4 .

[0119] The local brightness standard deviation reflects the degree of brightness fluctuation in the local range and can effectively measure the impact of contrast distortion on image quality. The extraction process of the local brightness standard deviation histogram feature is as follows:

[0120] G1: Use formula (8) to normalize the image;

[0121] G2: Divide the image into cells of the same size, calculate the standard deviation of each cell, and obtain the local brightness standard deviation feature map;

[0122] G3: Divide the local brightness standard deviation feature map into blocks of the same size, set the block size to 6*6 cells, and evenly divide the brightness standard deviation of each block into 10 histogram channels to obtain the local brightness standard deviation histogram H 5 .

[0123] In step S3 of this embodiment, the running status evaluation includes the image quality score and the monitoring device quality. The monitoring device quality is obtained by distinguishing the image distortion type. The specific method for distinguishing the image distortion type is as follows:

[0124] The BRISQUE natural image feature algorithm is selected. First, the block MSCN feature matrix of the image is calculated, the block MSCN distribution is calculated, and two parameters are obtained by fitting with a symmetric Gaussian curve as two eigenvalues. Then the matrix of the inner product of the image block MSCN in four directions is calculated, and the distribution is fitted with an asymmetric Gaussian curve to obtain four parameters, a total of 16 features are obtained. Finally, the image is reduced by half, and the first two processes are repeated to obtain a total of 36 eigenvalues, which are the BRISQUE image features. Five high-quality surveillance images of different scenes and different times are taken. A distortion simulation algorithm is used. For each high-quality surveillance image, 500 distorted surveillance images with 5 distortion types and 10 distortion levels are generated as the distortion surveillance image database and the distortion type labels of the human eye calibration data set images are calculated. The BRISQUE features of each image are calculated. For each feature, SVM is used to train a classifier that maps the image feature to whether the distortion type exists. This process is repeated for the five distortion types studied to obtain classifiers for whether the five distortion types exist or not. The image distortion type is distinguished by the classifier.

[0125] In step S3 of this embodiment, the operation status evaluation of the collected monitoring image specifically includes:

[0126] A: Extract image features: Since the brightness distribution of pixels in natural images basically conforms to the Gaussian distribution diagram, the image is divided into blocks, and the pixels in each block are normalized and Gaussian weighted to obtain the block MSCN value. The original image can thus obtain the block MSCN matrix, which can be fitted with a symmetric Gaussian distribution to obtain two parameters as eigenvalues. In addition, since there is a correlation between adjacent blocks, the block MSCN matrix of the inner product matrix of adjacent blocks in four directions is calculated, and the asymmetric Gaussian distribution is fitted respectively to obtain 16 eigenvalues. After the image is reduced by half, the above eigenvalues ​​are calculated again, and a total of 36 parameter eigenvalues ​​are obtained as image features.

[0127] B: Generate distortion degree labels: Consider the types of distortion that may be caused by camera failure, and infer the fault conditions based on the distortion degrees of different distortion types. The distortion simulation function is implemented through code. Blur distortion is implemented through different filter convolutions. Overbrightness, overdarkness, high and low contrast, and noise distortion are all implemented using MATLAB's built-in functions. Block distortion randomly generates square occlusions based on the given number and size of blocks. For each original image, the distortion type, distortion degree, and number of mixed distortions are randomly given to generate 100 distorted images. For each of the 500 distorted images, the human eye is used to determine whether the above types of distortion exist, and the human eye calibration results for each image are derived. The 500 images are randomly divided into test sets and training sets. For each distortion type, the image features of each image in the training set are fitted with the average distortion degree of the distortion type. The regression model is generated using SVR, and the SROCC value is predicted and calculated on the test set. The parameters are adjusted until the SROCC reaches 0.9.

[0128] C: Calculate the BRISQUE features of each image, and for each feature, use SVM to train a classifier that maps the image feature to the presence or absence of the distortion type. Repeat this process for the five distortion types studied to obtain classifiers for the presence or absence of the five distortion types.

[0129] The change of the image operation status in step S4 of this embodiment includes the change of the image quality score and the quality of the monitoring device, wherein the display method of the change of the image quality score is:

[0130] Several quality scores are read from the database and displayed on the interface in the form of a line graph, with the horizontal axis being time and the vertical axis being the image quality score. The final monitoring image quality evaluation is shown as follows: Figure 5 As shown;

[0131] The quality of the monitoring equipment is displayed as follows: for images with a quality score higher than the threshold, it is judged to be problem-free, and the monitoring equipment is displayed as excellent; for images with a quality score lower than the threshold, it is judged to have problems, and the specific type of image distortion is displayed, judging whether it is mosaic, color distortion, large image noise, etc., and the judgment result is output, indicating that the monitoring equipment has problems. The final equipment quality situation is as follows: Figure 6 shown.

[0132] In order to verify the performance of the method of the present invention, the performance of the method of the present invention (MFD) and other methods on the exBeDDE database are compared in this example. The results are shown in Table 1, where the optimal and suboptimal performance values ​​corresponding to different evaluation indicators are highlighted in bold;

[0133] Table 1

[0134]

[0135] The performance comparison results of MFD and other methods on classic databases are shown in Table 2:

[0136] Table 2

[0137]

[0138]

[0139] The mean and variance results of the performance of MFD and other methods on the classic database are shown in Table 3:

[0140] Table 3

[0141]

[0142] It can be seen from the comparative data in Tables 1 to 3 that the method of the present invention (MFD) can accurately evaluate the quality of the fogged image, is applicable to different distortion types, and has a high consistency with the subjective evaluation results of the human eye.

Claims

1. A monitoring operation monitoring method based on image quality, It is characterized in that The steps include: S1: Collect surveillance images; S2: Perform quality evaluation on the collected monitoring images by constructing the image quality score evaluation model to obtain the quality evaluation image quality score; S3: Evaluate the operating status of the collected monitoring images according to the image quality score; S4: Real-time dynamic display of changes in the operating status of monitoring images; S5: monitor the monitoring operation status according to the monitoring image operation status; The method for constructing the image quality score evaluation model in step S2 is: B1: Extract five image quality related features, namely, oriented gradient histogram feature, local gradient intensity histogram feature, local brightness histogram feature, local gradient intensity standard deviation histogram feature and local brightness standard deviation histogram feature; B2: Calculate five feature maps of the reference image and the distorted image based on the extracted features, and calculate the difference between the feature maps of the reference image and the distorted image to obtain a difference feature map; B3: The mean of the difference feature map is used to represent the different characteristic differences between the reference image and the distorted image, and the difference values ​​corresponding to the five feature maps of each distorted image are fused to obtain the difference vector; B4: The image quality score evaluation model is obtained by learning the mapping relationship between the difference vector and the image quality score through SVR; The method for extracting the five image quality related features in step B1 is: The steps of directional gradient histogram feature extraction are as follows: C1: Use formula (1) to perform gamma correction on the image; In the formula, R(i,j), G(i,j) and B(i,j) correspond to the RGB color channel values ​​of the image at (i,j), and I(i,j) is the grayscale value of the pixel at (i,j) after correction. C2: Use formula (2) and formula (3) to calculate the horizontal gradient and vertical gradient strength of the image at (i, j) respectively; In the formula, Gx(i,j) and Gy(i,j) correspond to the horizontal gradient and vertical gradient at (i,j) respectively. G x (i,j)=I(i+1,j)-I(i-1,j) (2) G y (i,j)=I(i,j+1)-I(i,i-1) (3) C3: Calculate the gradient intensity value G(i,j) and the gradient direction θ of the image at (i,j) using the horizontal gradient and vertical gradient obtained in step C2. The specific calculation method is shown in formula (4) and formula (5). Since the gradient direction will be divided into quadrants according to the positive and negative angles of Gx(i,j) and θ, the range of θ is [0,360°]; C4: Divide the image into non-overlapping connected areas of the same size and define them as cells. Divide the image into cells of the same size and use the weighted method to calculate the gradient direction histogram of each cell. The gradient direction of each cell is evenly divided into 9 histogram channels. The weight is the gradient intensity corresponding to the gradient direction, and the directional gradient histogram is obtained. C5: The histogram of each cell is normalized using formula (6), where hist i is the i-th value in the histogram vector corresponding to a single cell, HIST i is the i-th value in the histogram vector corresponding to the normalized cell; define a 2*2 cell area as a block, connect the histograms of the 4 cells in each block to obtain a feature vector with a length of 36, and define the directional gradient histogram after block normalization as H 1 ; The local gradient intensity histogram feature extraction process is as follows: D1: gamma correction of the image is performed using formula (1); D2: Use formula (2) and formula (3) to calculate the gradient strength of each pixel in the image in the horizontal and vertical directions respectively, and use formula (4) to obtain the gradient strength feature map of the image; D3: Use formula (7) to normalize the local gradient intensity feature map of the image; In the formula, G(i, j) represents the gradient intensity at (i, j), G min and G max represent the maximum and minimum values of the local gradient intensity feature maps of the reference image and the distorted image, G n (i, j) represents the value at (i, j) after normalization; D4: Divide the image into cells of the same size, divide the gradient intensity of each cell into 10 histogram channels and calculate the gradient intensity histogram H of each cell 2 ; The extraction process of local brightness histogram features is as follows: E1: Normalize the brightness of the reference image and the distorted image; The normalization formula is shown in (8), where I(i,j) represents the pixel value at (i,j), I min and I max Indicates the maximum and minimum brightness of the reference image and the distorted image, I n (i,j) represents the value at (i,j) after normalization; E2: Divide the image into cells of the same size, divide the brightness of each cell into 10 histogram channels, and obtain the local brightness histogram H 3 ; The calculation process of the local gradient intensity standard deviation is as follows: F1: gamma correction of the image using formula (1); F2: Use formula (2) and formula (3) to calculate the gradient strength of each pixel in the image in the horizontal and vertical directions respectively, and use formula (4) to obtain the gradient strength feature map of the image, and use formula (8) to realize the normalization of the local gradient strength feature map; F3: Divide the image into cells of the same size, calculate the standard deviation of each cell, and obtain the local gradient intensity standard deviation feature map; F4: Divide the local gradient intensity standard deviation feature map into blocks of the same size. The size of each block is set to 6*6 cells. Divide the gradient intensity standard deviation of each block into 10 histogram channels on average. Statistically obtain the local gradient intensity standard deviation histogram H 4 ; The extraction process of local brightness standard deviation histogram features is as follows: G1: Use formula (8) to normalize the image; G2: Divide the image into cells of the same size, calculate the standard deviation of each cell, and obtain the local brightness standard deviation feature map; G3: Divide the local brightness standard deviation feature map into blocks of the same size. Set the block size to 6 * 6 cells, and evenly divide the brightness standard deviation of each block into 10 histogram channels to obtain the local brightness standard deviation histogram H 5 .

2. The method for monitoring operation based on image quality according to claim 1, It is characterized in that In step S1, the real-time acquisition and transmission of monitoring images are realized through the PC host, the Raspberry Pi mainboard and the USB camera, which specifically includes the following: A1: Burn MATLAB image on SD card: Connect the USB camera to the Raspberry Pi mainboard, and operate the Raspberry Pi mainboard through MATLAB on the PC host to control the camera to collect video images in real time; first insert the SD card into the host card slot, open the MATLAB hardware resource package, select MATLAB Simulink Raspberry Pi hardware support package to download and install, burn the MATLAB image to the SD card, select WIFI connection for the connection between MATLAB and the Raspberry Pi mainboard, enter the hotspot ID and password, and follow the steps to complete the wireless connection MATLAB image burning; A2: Connect MATLAB, Raspberry Pi mainboard and USB camera: Connect the USB camera to the USB2.0 port of the Raspberry Pi mainboard, insert the burned SD card into the card slot of the Raspberry Pi mainboard and turn on the computer, turn on the WIFI hotspot to connect the Raspberry Pi mainboard and the host to the same network, and enter re=raspi in the MATLAB command line to view the properties of the Raspberry Pi mainboard; A3: Use MATLAB to control the Raspberry Pi mainboard to collect images from the USB camera in real time: Open the Simulink module of MATLAB, create a Simulink file, find the Simulink support package for Raspberry Pi in the resource library, add the required components to build the model, connect the components, and start running real-time video image collection.

3. The monitoring operation monitoring method based on image quality according to claim 1, It is characterized in that The operation status evaluation in step S3 includes the image quality score and the monitoring device quality, wherein the monitoring device quality is obtained by distinguishing the image distortion type.

4. The method for monitoring operation based on image quality according to claim 3, It is characterized in that The specific method for determining the image distortion type in step S3 is: The BRISQUE natural image feature algorithm is selected. First, the block MSCN feature matrix of the image is calculated, the block MSCN distribution is calculated, and two parameters are obtained by fitting with a symmetric Gaussian curve as two eigenvalues. Then the matrix of the inner product of the image block MSCN in four directions is calculated, and the distribution is fitted with an asymmetric Gaussian curve to obtain four parameters, a total of 16 features are obtained. Finally, the image is reduced by half, and the first two processes are repeated to obtain a total of 36 eigenvalues, which are the BRISQUE image features. Five high-quality surveillance images of different scenes and different times are taken. A distortion simulation algorithm is used. For each high-quality surveillance image, 500 distorted surveillance images with 5 distortion types and 10 distortion levels are generated as the distortion surveillance image database and the distortion type labels of the human eye calibration data set images are calculated. The BRISQUE features of each image are calculated. For each feature, SVM is used to train a classifier that maps the image feature to whether the distortion type exists. This process is repeated for the five distortion types studied to obtain classifiers for whether the five distortion types exist or not. The image distortion type is distinguished by the classifier.

5. The method for monitoring operation based on image quality according to claim 1, It is characterized in that The change of the image running state in step S4 includes the change of the image quality score and the quality of the monitoring device, wherein the display method of the change of the image quality score is: Several quality scores are read from the database and displayed on the interface in the form of a line graph, with the horizontal axis being time and the vertical axis being the image quality score; The quality status of the monitoring device is displayed as follows: for images with a quality score higher than a threshold, it is judged that there is no problem, and the monitoring device displays it as excellent; for images with a quality score lower than a threshold, it is judged that there is a problem, and the specific image distortion type is displayed.

Images