System, method and device for evaluating and processing low-quality data for intelligent safety monitoring system, processor and medium thereof
Through deep learning technology, a multi-dimensional image quality evaluation system is built, which solves the problem of low-quality video image evaluation in intelligent security scenarios, and realizes accurate evaluation and robustness improvement of image quality, which is suitable for intelligent security systems.
Patent Information
- Application Number
- CN202411050982.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-22
AI Technical Summary
The prior art is difficult to effectively evaluate and process low-quality video images, especially in intelligent security scenarios, which leads to the impact of the accuracy of intelligent analysis tasks, and traditional methods cannot fully reflect the influencing factors of image quality on specific tasks.
Deep learning technology combined with the characteristics of intelligent analysis tasks is used to build a multi-dimensional and multi-level image quality evaluation system, including human body recognition module, objective quality analysis module and content quality analysis module. Through the comprehensive evaluation module, the weight is set according to different usage scenarios, and the images are evaluated. and judge human goals and content-level quality issues through deep learning models.
It realizes accurate evaluation of low-quality data, improves the robustness of the intelligent security system, avoids error analysis, saves computing resources, and is adapted to different application scenarios, with good versatility and practicality.
Smart Images

Figure CN119006987A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision, and in particular to the field of artificial intelligence, and specifically refers to a system, method, device, processor and readable storage medium thereof for implementing evaluation and processing of low-quality data for an intelligent safety monitoring system. Background Art
[0002] With the rapid development of artificial intelligence technology, intelligent video surveillance and security prevention have received extensive attention and application. In actual scenarios, due to the limitations of shooting equipment, environmental conditions and other factors, the quality of the collected video images is uneven, and there are a large number of low-quality images, which brings great challenges to subsequent intelligent analysis. Traditional image quality assessment methods are mainly based on the objective attributes of the image, such as clarity, contrast, noise, etc. These methods can describe the image quality from a macro perspective, but cannot reflect the quality influencing factors for specific tasks. For example, in security scenarios, the presence of factors such as complex background, human occlusion, and exceeding the boundaries of the screen will seriously interfere with the accuracy of intelligent analysis tasks such as behavior recognition and target detection.
[0003] In view of the impact of low-quality images on the performance of intelligent analysis, the current mainstream approach is to perform repair and enhancement processing on them, such as super-resolution reconstruction, denoising and deblurring. However, there are two major problems with this type of method: first, the enhancement process may introduce erroneous information, resulting in a result that "looks like but is not real"; second, the quality of the processed image still does not meet the minimum requirements of the algorithm. Therefore, before intelligent analysis, low-quality images that are not suitable as input are identified, and their quality levels are quantitatively evaluated and fed back to the analysis model so that it selectively rejects or reduces the response to low-quality inputs, which can improve the overall performance of the system, avoid erroneous analysis, and save computing resources.
[0004] However, there is still a lack of a set of low-quality image evaluation methods for security intelligent analysis. Most of the existing methods are based on shallow features and single dimensions, lacking content-level quality understanding and comprehensive evaluation capabilities. This invention aims to use deep learning technology, combined with the characteristics of intelligent analysis tasks, to build a comprehensive image quality evaluation system and form a standardized quality evaluation to guide and prompt subsequent intelligent analysis modules, which is of great significance and value in complex application scenarios. Summary of the invention
[0005] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and to provide a system, method, device, processor and readable storage medium thereof for evaluating and processing low-quality data for intelligent safety monitoring systems, which have good accuracy, good overall performance and a wide range of applications.
[0006] In order to achieve the above-mentioned purpose, the system, method, device, processor and readable storage medium thereof for evaluating and processing low-quality data for an intelligent safety monitoring system of the present invention are as follows:
[0007] The system for evaluating and processing low-quality data for an intelligent safety monitoring system has the following main features:
[0008] Human body recognition module, used to identify whether the video image contains human targets:
[0009] The objective quality analysis module is connected to the human body recognition module and is used to calculate and analyze the image quality indexes based on the image clarity, color, illumination, and noise angle using OpenCV to determine whether the image is of poor quality:
[0010] The content quality analysis module is connected to the objective quality analysis module and is used to evaluate the image content.
[0011] Preferably, the system also includes a comprehensive evaluation module, which includes the human body recognition module, the objective quality analysis module and the content quality analysis module, and also includes a weight module, the input ends of the objective quality analysis module and the content quality analysis module are connected to the human body recognition module, and the output ends of the objective quality analysis module and the content quality analysis module are connected to the weight module; the comprehensive evaluation module is used to set the weight of each dimension according to different usage scenarios and comprehensively evaluate the image quality.
[0012] Preferably, the construction of the human body recognition module includes the following steps:
[0013] Construct a dataset, obtain raw data from surveillance cameras in real scenes, use an open source human detection model to extract regions of interest containing human bodies as positive samples, and regions not containing human bodies as negative samples, and divide the data into training, validation, and test sets.
[0014] Build and train the model, using the ResNet50 model under the PaddleCLS deep learning framework for binary classification training;
[0015] Conduct model testing and iterative update and adjustment.
[0016] Preferably, the objective quality analysis module uses Laplace transform to detect the degree of image blur and analyzes the clarity of the image; the objective quality analysis module analyzes the color of the image by calculating the distribution and intensity evaluation of the colors in the image; the objective quality analysis module analyzes the illumination of the image by calculating the average brightness evaluation of the image; the objective quality analysis module analyzes the noise of the image by calculating the signal-to-noise ratio of the image.
[0017] Preferably, the content quality analysis module adopts a deep learning classification model to identify content-level quality issues in the input image that are closely related to intelligent analysis, constructs four types of data sets of normal images, out-of-boundary images, occluded images and abnormally distorted images by performing data enhancement operations on normal images, and trains a multi-classification model based on the ResNet50 backbone network.
[0018] Preferably, the construction of the content quality analysis module includes the following steps:
[0019] Construct a data set to collect and divide four types of data into normal images, super-boundary images, occluded images, and abnormally distorted images:
[0020] Use the ResNet50 model under the PaddleCLS deep learning framework for four-category training;
[0021] Conduct model testing and iterative update and adjustment.
[0022] Preferably, the comprehensive evaluation module integrates the output results of the human recognition module, the objective quality analysis module and the content quality analysis module, and weightedly summarizes and normalizes them through a weight module.
[0023] The method for evaluating and processing low-quality data for an intelligent safety monitoring system using the above system is mainly characterized in that the method comprises the following steps:
[0024] (1) Construct a human recognition module to identify video images and determine whether they contain human targets:
[0025] (2) The objective quality analysis module uses OpenCV to analyze the image quality indicators and determine whether it is a poor quality image:
[0026] (3) Build a content quality analysis module to evaluate image content:
[0027] (4) The comprehensive evaluation module sets the weight of each dimension according to different usage scenarios and comprehensively evaluates the image quality.
[0028] Preferably, the step (1) of constructing a human body recognition module to recognize a body in a video image comprises the following steps:
[0029] (1.1) Construct a dataset, obtain raw data from surveillance cameras in real scenes, use an open source human detection model to extract regions of interest containing human bodies as positive samples, and regions not containing human bodies as negative samples, and divide the data into training set, validation set, and test set;
[0030] (1.2) Build and train the model, using the ResNet50 model under the PaddleCLS deep learning framework for binary classification training;
[0031] (1.3) Conduct model testing and iterative updates and adjustments.
[0032] Preferably, the step (2) specifically comprises the following steps:
[0033] (2.1) Detect the blur level of the image through Laplace transform;
[0034] (2.2) Evaluate whether the image has color distortion by calculating the distribution and intensity of colors in the image;
[0035] (2.3) Evaluate the lighting conditions by calculating the average brightness of the image;
[0036] (2.4) The noise in the image is evaluated by calculating the signal-to-noise ratio of the image.
[0037] Preferably, the step (3) of constructing the content quality analysis module specifically includes the following steps:
[0038] (3.1) Construct a data set to collect and divide the data into four categories: normal images, out-of-boundary images, occluded images, and abnormally distorted images:
[0039] (3.2) Use the ResNet50 model under the PaddleCLS deep learning framework to perform four-category training;
[0040] (3.3) Conduct model testing and iterative update and adjustment.
[0041] Preferably, the step (4) specifically comprises the following steps:
[0042] (4.1) Processing the input image through the human body recognition module;
[0043] (4.2) Determine whether there is a human target in the image. If yes, proceed to step (4.3); otherwise, proceed to step (4.4);
[0044] (4.3) Analyzing the image through an objective quality analysis module and a content quality analysis module;
[0045] (4.4) Calculate the comprehensive quality evaluation index Q;
[0046] (4.5) Based on the comprehensive quality evaluation index Q and the confidence level P of the output results of the artificial intelligence model AI , and obtain the final decision.
[0047] The main feature of the device for realizing evaluation and processing of low-quality data for an intelligent safety monitoring system is that the device comprises:
[0048] a processor configured to execute computer executable instructions;
[0049] A memory stores one or more computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the above-mentioned method for evaluating and processing low-quality data for an intelligent safety monitoring system are implemented.
[0050] The processor for implementing evaluation and processing of low-quality data for an intelligent safety and monitoring system has the main feature that the processor is configured to execute computer-executable instructions. When the computer-executable instructions are executed by the processor, the various steps of the above-mentioned method for implementing evaluation and processing of low-quality data for an intelligent safety and monitoring system are implemented.
[0051] The main feature of the computer-readable storage medium is that a computer program is stored thereon, and the computer program can be executed by a processor to implement the various steps of the above-mentioned method for evaluating and processing low-quality data for an intelligent safety monitoring system.
[0052] The system, method, device, processor and readable storage medium thereof for evaluating and processing low-quality data for an intelligent safety monitoring system of the present invention are adopted. Aiming at the special needs of image quality assessment in the field of intelligent security, a multi-dimensional and multi-level analysis method is proposed to comprehensively examine the quality risks of input images, and the evaluation results are more accurate and reliable. A deep learning method is used to determine the existence of human targets and identify content-level quality defects, overcoming the limitation that traditional methods cannot understand image semantic information. Comprehensive quality scores are used to guide subsequent analysis modules to make reasonable responses, and the robustness of the system in the face of low-quality inputs is significantly improved without increasing computing overhead, thus avoiding erroneous analysis and saving invalid calculations. The evaluation modules and scoring strategies can be flexibly customized to adapt to different application scenarios, and have good versatility and practicality. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] Figure 1 The present invention is a flowchart of the overall architecture of a method for evaluating and processing low-quality data for an intelligent safety monitoring system.
[0054] Figure 2The figure is a schematic diagram of the working process of the human body recognition module of the system for evaluating and processing low-quality data in the intelligent safety monitoring system of the present invention.
[0055] Figure 3 The present invention is a schematic diagram of the workflow of an objective quality analysis module based on OpenCV for a system for evaluating and processing low-quality data for an intelligent safety monitoring system.
[0056] Figure 4 The figure is a schematic diagram of the workflow of the neural network-based image quality analysis module of the system for evaluating and processing low-quality data for the intelligent safety monitoring system of the present invention.
[0057] Figure 5 Various low-quality image examples for an intelligent safety monitoring system that implements a system for evaluating and processing low-quality data for an intelligent safety monitoring system of the present invention.
[0058] Figure 6 The figure is a schematic diagram of the workflow of the comprehensive scoring module of the system for evaluating and processing low-quality data for the intelligent safety monitoring system of the present invention. DETAILED DESCRIPTION
[0059] In order to more clearly describe the technical content of the present invention, further description is given below in conjunction with specific embodiments.
[0060] The system for evaluating and processing low-quality data for the intelligent safety monitoring system of the present invention includes:
[0061] Human body recognition module, used to identify whether the video image contains human targets:
[0062] The objective quality analysis module is connected to the human body recognition module and is used to calculate and analyze the image quality indexes based on the image clarity, color, illumination, and noise angle using OpenCV to determine whether the image is of poor quality:
[0063] The content quality analysis module is connected to the objective quality analysis module and is used to evaluate the image content.
[0064] As a preferred embodiment of the present invention, the system also includes a comprehensive evaluation module, which includes the human body recognition module, the objective quality analysis module and the content quality analysis module, and also includes a weight module. The input ends of the objective quality analysis module and the content quality analysis module are connected to the human body recognition module, and the output ends of the objective quality analysis module and the content quality analysis module are connected to the weight module; the comprehensive evaluation module is used to set the weight of each dimension according to different usage scenarios and comprehensively evaluate the image quality.
[0065] As a preferred embodiment of the present invention, the construction of the human body recognition module includes the following steps:
[0066] Construct a dataset, obtain raw data from surveillance cameras in real scenes, use an open source human detection model to extract regions of interest containing human bodies as positive samples, and regions not containing human bodies as negative samples, and divide the data into training, validation, and test sets.
[0067] Build and train the model, using the ResNet50 model under the PaddleCLS deep learning framework for binary classification training;
[0068] Conduct model testing and iterative update and adjustment.
[0069] As a preferred embodiment of the present invention, the objective quality analysis module uses Laplace transform to detect the degree of image blur and analyzes the clarity of the image; the objective quality analysis module analyzes the color of the image by calculating the distribution and intensity evaluation of the colors in the image; the objective quality analysis module analyzes the illumination of the image by calculating the average brightness evaluation of the image; the objective quality analysis module analyzes the noise of the image by calculating the signal-to-noise ratio of the image.
[0070] As a preferred embodiment of the present invention, the content quality analysis module adopts a deep learning classification model to identify content-level quality issues in the input image that are closely related to intelligent analysis. By performing data enhancement operations on normal images, four data sets of normal images, super-boundary images, occluded images and abnormally distorted images are constructed, and a multi-classification model based on the ResNet50 backbone network is trained.
[0071] As a preferred embodiment of the present invention, the construction of the content quality analysis module includes the following steps:
[0072] Construct a data set to collect and divide four types of data into normal images, super-boundary images, occluded images, and abnormally distorted images:
[0073] Use the ResNet50 model under the PaddleCLS deep learning framework for four-category training;
[0074] Conduct model testing and iterative update and adjustment.
[0075] As a preferred embodiment of the present invention, the comprehensive evaluation module integrates the output results of the human recognition module, the objective quality analysis module and the content quality analysis module, and weightedly summarizes and normalizes them through a weight module.
[0076] The present invention uses the above system to implement a method for evaluating and processing low-quality data for an intelligent safety monitoring system, wherein the method comprises the following steps:
[0077] (1) Construct a human recognition module to identify video images and determine whether they contain human targets:
[0078] (2) The objective quality analysis module uses OpenCV to analyze the image quality indicators and determine whether it is a poor quality image:
[0079] (3) Build a content quality analysis module to evaluate image content:
[0080] (4) The comprehensive evaluation module sets the weight of each dimension according to different usage scenarios and comprehensively evaluates the image quality.
[0081] As a preferred embodiment of the present invention, the step (1) of constructing a human body recognition module to recognize a video image body comprises the following steps:
[0082] (1.1) Construct a dataset, obtain raw data from surveillance cameras in real scenes, use an open source human detection model to extract regions of interest containing human bodies as positive samples, and regions not containing human bodies as negative samples, and divide the data into training set, validation set, and test set;
[0083] (1.2) Build and train the model, using the ResNet50 model under the PaddleCLS deep learning framework for binary classification training;
[0084] (1.3) Conduct model testing and iterative updates and adjustments.
[0085] As a preferred embodiment of the present invention, the step (2) specifically comprises the following steps:
[0086] (2.1) Detect the blur level of the image through Laplace transform;
[0087] (2.2) Evaluate whether the image has color distortion by calculating the distribution and intensity of colors in the image;
[0088] (2.3) Evaluate the lighting conditions by calculating the average brightness of the image;
[0089] (2.4) The noise in the image is evaluated by calculating the signal-to-noise ratio of the image.
[0090] As a preferred embodiment of the present invention, the step (3) of constructing the content quality analysis module specifically includes the following steps:
[0091] (3.1) Construct a data set to collect and divide the data into four categories: normal images, out-of-boundary images, occluded images, and abnormally distorted images:
[0092] (3.2) Use the ResNet50 model under the PaddleCLS deep learning framework to perform four-category training;
[0093] (3.3) Conduct model testing and iterative update and adjustment.
[0094] As a preferred embodiment of the present invention, the step (4) specifically comprises the following steps:
[0095] (4.1) Processing the input image through the human body recognition module;
[0096] (4.2) Determine whether there is a human target in the image. If yes, proceed to step (4.3); otherwise, proceed to step (4.4);
[0097] (4.3) Analyzing the image through an objective quality analysis module and a content quality analysis module;
[0098] (4.4) Calculate the comprehensive quality evaluation index Q;
[0099] (4.5) Based on the comprehensive quality evaluation index Q and the confidence level P of the output results of the artificial intelligence model AI , and obtain the final decision.
[0100] The device for implementing evaluation and processing of low-quality data for an intelligent safety monitoring system of the present invention comprises:
[0101] a processor configured to execute computer executable instructions;
[0102] A memory stores one or more computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the above-mentioned method for evaluating and processing low-quality data for an intelligent safety monitoring system are implemented.
[0103] The processor of the present invention is used to implement evaluation and processing of low-quality data for an intelligent safety and monitoring system, wherein the processor is configured to execute computer-executable instructions. When the computer-executable instructions are executed by the processor, the various steps of the above-mentioned method for implementing evaluation and processing of low-quality data for an intelligent safety and monitoring system are implemented.
[0104] The computer-readable storage medium of the present invention stores a computer program thereon, and the computer program can be executed by a processor to implement the various steps of the above-mentioned method for evaluating and processing low-quality data for an intelligent safety monitoring system.
[0105] In a specific embodiment of the present invention, a low-quality data evaluation system and method for an intelligent safety monitoring system is provided, which obtains a comprehensive score by analyzing the image quality in multiple dimensions and multiple levels, so as to guide the subsequent intelligent analysis module to reasonably process input data of different qualities and improve the overall performance of the system.
[0106] The construction of the human body recognition module includes the following steps:
[0107] Collect and annotate data, and use the open source human detection model and manual annotation to divide the original data into two categories: image data containing human targets and image data excluding human targets:
[0108] Use the labeled data to train the resnet50 binary classification model under the paddlecls framework:
[0109] Use the trained model to identify whether the image contains a human body.
[0110] The specific steps of the neural network-based content quality analysis module include:
[0111] Collect four types of data: normal images, out-of-boundary images, occluded images, and abnormally distorted images:
[0112] At the same time, it is also necessary to convert normal images into the above four types of images through data enhancement:
[0113] Using the converted image data, a multi-classification model is trained under the resnet50 framework, and the model is updated after multiple repeated iterations and distillations.
[0114] In the image quality comprehensive scoring module, the weight of each dimension is set and adjusted according to different usage scenarios and is finally determined through experimental verification.
[0115] The system can be integrated into any artificial intelligence analysis system, and includes the following steps:
[0116] Get the AI model's analysis results and confidence level for the image:
[0117] Image quality scores were obtained using the low-quality data evaluation system:
[0118] Multiply the confidence of the AI model output by the image quality score. When the score is close to 0, it indicates that the image quality is poor, which lowers the confidence of the model output. When the score is close to 1, it has no effect on the model output.
[0119] Specifically, the low-quality data evaluation system proposed in the present invention includes the following modules:
[0120] Human recognition module: uses a deep learning classification model to determine whether the input image contains human targets.
[0121] Furthermore, the construction of the human body recognition module includes the following steps:
[0122] The dataset is constructed by obtaining raw data from surveillance cameras in actual scenes. An open source human detection model is used to extract regions of interest containing human bodies as positive samples and regions not containing human bodies as negative samples. The data is divided into training set, validation set, and test set.
[0123] For model construction and training, we use the ResNet50 model under the PaddleCLS deep learning framework, which includes a feature extraction module, a global average pooling module, a fully connected layer, and a Softmax layer for binary classification training.
[0124] Model testing and iterative update and adjustment.
[0125] Objective quality analysis module: Using the OpenCV library, calculate the traditional quality indicators of the image from the perspectives of clarity, color, lighting, and noise, and preliminarily determine whether the image is of poor quality.
[0126] Furthermore, the clarity analysis uses Laplace transform to detect the blur of the image; the color richness is evaluated by calculating the distribution and intensity of the colors in the image; the lighting condition is evaluated by calculating the average brightness of the image; and the noise can be quantified by calculating the signal-to-noise ratio (SNR) of the image. The analysis results of the four dimensions are normalized to obtain the corresponding indicators to indicate the image quality.
[0127] Content quality analysis module: Deep learning classification models are further used to identify content-level quality issues in input images that are closely related to intelligent analysis, such as human bodies exceeding boundaries and severe occlusion. By performing data enhancement operations on normal images, four types of data sets, "normal, beyond boundaries, occlusion, and abnormal interference", are constructed, and a multi-classification model based on the ResNet50 backbone network is trained.
[0128] Furthermore, the construction of the content quality analysis module includes the following steps:
[0129] The dataset is constructed to collect and divide four types of data: normal images, super-boundary images, occluded images, and abnormal distorted images. Low-quality images can be generated from normal images through data enhancement methods, such as randomly intercepting edge areas to generate super-boundary images, pasting objects in the segmentation image on the normal human body image to construct occluded images, and using OpenCV tools to artificially synthesize abnormal interference images.
[0130] Model construction and training, using the ResNet50 model under the PaddleCLS deep learning framework for four-category training.
[0131] Model testing and iterative update adjustment. This module outputs the category and confidence information of the image.
[0132] Comprehensive evaluation module: This module consists of the above three modules and a weight module, and the weighted sum is used to obtain the comprehensive quality score of the image.
[0133] Furthermore, the output results of the above three modules are integrated, weighted and normalized through the weight module to obtain an image quality score between 0 and 1. The closer the quality score is to 0, the worse the quality is, and the closer it is to 1, the better the quality is. Multiply the quality score by the confidence of the output result of the artificial intelligence model to get the final decision result. The better the quality of the image, the more reliable the result obtained by the artificial intelligence model; the worse the quality of the image, the confidence of the result will be lowered to close to 0 by the quality score, reducing the degree to which the model is affected by image quality and improving the robustness of the model. The setting of weights needs to be determined through experimental verification according to different usage scenarios. The weights are determined through experimental data analysis and verification, and can be flexibly adjusted according to different needs.
[0134] In the business process of intelligent video analysis, the low-quality data evaluation system of the present invention can be deployed as a front-end module, and the output comprehensive quality score is multiplied by the output result of the subsequent intelligent analysis model to suppress the erroneous response of low-quality input data. When the comprehensive quality score is close to 0, it means that the input image quality is extremely poor and is not suitable for subsequent analysis, and the corresponding model output confidence will be greatly reduced: when the comprehensive quality score is close to 1, it means that the input image quality is good and will not affect the credibility of the subsequent analysis results.
[0135] like Figure 1 As shown, this system includes four modules: human recognition module, objective quality analysis module, content quality analysis module, and comprehensive evaluation module.
[0136] like Figure 2 As shown in the figure, the human recognition module determines whether the image contains human targets based on the ResNet50 classification model.
[0137] like Figure 3 As shown, the objective quality analysis module analyzes image quality from four dimensions: clarity, color distortion, lighting, and noise.
[0138] like Figure 4 As shown in the figure, the image quality analysis module uses ResNet50 to classify images into four categories: normal images, over-boundary images, occluded images, and abnormally distorted images.
[0139] like Figure 5 As shown, various low-quality images include overexposure, low brightness, occlusion, and exceeding boundaries.
[0140] like Figure 6As shown, the comprehensive scoring module is deployed after the artificial intelligence model. The weights of different dimensions are set according to the usage scenario, and the results of the three analysis modules are weighted and scored. The scoring result is multiplied by the output of the artificial intelligence model to obtain the final decision.
[0141] like Figure 1 As shown, the embodiment of the present invention proposes a low-quality data evaluation system and method for an intelligent safety monitoring system, including the following modules:
[0142] The human recognition module uses a deep learning classification model to determine whether the input image contains a human target;
[0143] The objective quality analysis module uses the OpenCV library to calculate the traditional quality indicators of the image from the perspectives of clarity, color, lighting, noise, etc., and preliminarily determines whether the image is of poor quality;
[0144] The content quality analysis module further uses a deep learning classification model to identify content-level quality issues in the input image that are closely related to intelligent analysis, such as human bodies exceeding the boundary, severe occlusion, and abnormal distortion;
[0145] The comprehensive evaluation module consists of the above three modules and a weight module. The weighted sum is used to obtain the comprehensive quality score of the image. The quality score is multiplied by the confidence level of the output result of the artificial intelligence model to obtain the final decision result.
[0146] The following lists the preferred embodiments of the low-quality data evaluation system and method for intelligent safety monitoring systems in industrial safety monitoring systems to clearly illustrate the contents of the present invention. It should be clear that the contents of the present invention are not limited to the following embodiments, and other improvements through conventional technical means of ordinary technicians in this field are also within the scope of the concept of the present invention.
[0147] S100, build a human recognition module (see Figure 2 ), a classification model based on the ResNet-50 neural network is used to identify whether the video image contains human targets. This module is mainly divided into the following steps:
[0148] S101 training data preparation, obtain actual scene data from surveillance cameras, use the open source human detection model to crop the area of interest containing the human body as the positive sample, and the area not containing the human body as the negative sample, see Figure 5 (a) The data is divided into a training set (accounting for 80% of the total), a validation set (accounting for 10% of the total) and a test set (accounting for 10% of the total).
[0149] S102 model construction and training, using the ResNet-50 model under the PaddlePaddle deep learning framework, including feature extraction module, global average pooling module, fully connected layer and Softmax layer for binary classification training. The loss function uses the cross entropy loss function, and the optimization algorithm uses Adam.
[0150] Specifically, the input image I is extracted through the Resnet convolution layer K to obtain the feature map F:
[0151] F = I * K (a);
[0152] Where * represents the convolution operation.
[0153] Then perform global average pooling on the feature map F to obtain a one-dimensional feature vector v:
[0154]
[0155] Among them, H and W represent the height and width of the feature map respectively, and F ij Represents the value of the feature map at position (i, j). Then the feature vector v is passed through the fully connected layer (weight matrix W and bias vector b) to obtain the output vector z before classification:
[0156] z=Wv+b(c);
[0157] The output vector z is converted into a classification probability vector through the Softmax function
[0158]
[0159] In the formula, is the predicted probability of the ith class, z i is the i-th element in z.
[0160] In summary, the whole process can be expressed as follows:
[0161]
[0162] S103, model testing and iterative update: Use the test set data to test the model effect. If the effect is not ideal, analyze the reasons, supplement the data and retrain the model until the model effect meets the expected requirements.
[0163] S200, build an objective quality analysis module, see Figure 3 This module mainly uses OpenCV tools to analyze traditional image quality indicators, including clarity, color distortion, lighting and noise. Figure 5(b). Combining these indicators, determine whether the image is a poor quality image in the traditional sense. The specific implementation is as follows:
[0164] S201. It is a common method to detect the blur degree of an image through Laplace transform. This method is based on the calculation of the second-order derivative of the image by the Laplace operator to capture the sharp features in the image. The high-frequency components of the blurred image are reduced, resulting in a smaller Laplace transform result. Assuming that the input image is represented by I, whose size is M×N, the image I is Laplace transformed to obtain the Laplace transformed image ΔI. In order to quantify the blur degree, we calculate the absolute value average of the Laplace transformed image, which is defined as the blur evaluation index L
[0165]
[0166] Where: M and N are the number of rows and columns of the image, respectively. ΔI(i, j) is the pixel value at (i, j) after Laplace transform. A larger L value indicates that there are more high-frequency components (i.e., edges and details) in the image, so the image is clearer. A smaller L value indicates that there are fewer high-frequency components in the image, and the edges and details are not obvious, so the image is blurry.
[0167] S202: It is a common method to evaluate whether the image has color distortion by calculating the distribution and intensity of the colors in the image. This method determines whether the color of the image is normal based on the statistical characteristics of the color channel.
[0168] Specifically, assume that the input image is represented by I, whose size is M×N and is a color image. Image I can be represented as a combination of three color channels: red channel R, green channel G, and blue channel B. The three channels can be combined into a three-dimensional matrix I(i, j, k), where k∈{R, G, B} represents the color channel. We can use vectorized operations to calculate the mean μ of each color channel k and standard deviation σ k :
[0169]
[0170] Where k is R, G and B respectively, we can define the mean vector μ and the standard deviation vector σ:
[0171] μ=[μ R , μ G , μ B ],σ=[σ R , σ G , σ B ] (h);
[0172] Then the norms of the mean vector and the standard deviation vector are calculated as the two parts of the color distortion evaluation index. The final color distortion evaluation index C can be defined as the weighted sum of the above two norms:
[0173]
[0174] Where α and β are weight factors used to balance the impact of mean difference and standard deviation difference. The average of the three channel means, It is the average value of the standard deviation of the three channels. A larger C value indicates that the mean and standard deviation of each color channel in the image are quite different, indicating that the image has color distortion. A smaller C value indicates that the mean and standard deviation of each color channel in the image are relatively small, indicating that the image color is normal.
[0175] S203 , the lighting condition is evaluated by calculating the average brightness of the image.
[0176] Specifically, assume that the input image I is a color image with a size of M×N and contains three color channels: the red channel I R 、Green Channel I G and blue channel I B The brightness of an image can be obtained by weighted summing the three color channels. The commonly used brightness calculation formula is:
[0177] I Y (i, j) = 0.299·I R (i,j)+0.587·I G (i,j)+0.114·I B (i, j) (J);
[0178] Among them I Y (i, j) represents the brightness value of the image at position (i, j). The average brightness B of the entire image can be obtained by averaging the brightness values of all pixels.
[0179]
[0180] A larger B value indicates that the image is brighter overall, indicating that the image has good lighting conditions and may have been taken in a bright environment. A smaller B value indicates that the image is darker overall, indicating that the image has poor lighting conditions and may have been taken in a dim environment.
[0181] S204: The noise in the image can be effectively evaluated by calculating the signal-to-noise ratio (SNR) of the image. Assuming that the input image is represented by I, the following is a detailed description of how to calculate and obtain the final evaluation index S and an explanation of the meaning of the index.
[0182] Specifically, assume that the input image I is a grayscale image with a size of M×N. If the image is a color image, it can be converted to a grayscale image first. The signal strength of the image can be calculated by calculating the mean μ of all pixel values. I get:
[0183]
[0184] The noise intensity of an image can be obtained by calculating the standard deviation σI of all pixel values:
[0185]
[0186] The signal-to-noise ratio S is defined as the ratio of signal strength to noise strength. It is usually expressed in decibels (dB):
[0187]
[0188] A higher S value means that the signal strength in the image is much greater than the noise strength, indicating that the image quality is good and there is less noise. A lower S value means that the signal strength in the image is close to or lower than the noise strength, indicating that the image quality is poor and there is more noise.
[0189] S300, build a content quality analysis module (see Figure 4 ), a classification model based on the ResNet-50 neural network is used to evaluate the image content and identify whether the human body exceeds the boundary, whether there is severe occlusion, or abnormal distortion. The main steps are as follows:
[0190] S301. Training data preparation. In order to train a multi-classification model that can identify normal images, out-of-boundary images, occluded images, and abnormal interference images, we need to collect and prepare the corresponding data. Figure 5 (c). This part of data can be artificially constructed by data enhancement. The specific steps are as follows: collect normal human data. First, collect a large amount of normal human image data; generate super-boundary images and use OpenCV tools to crop normal images. Perform horizontal or vertical cropping from random positions in the middle of the image to obtain super-boundary images of varying degrees; generate occluded images. Select object images from public segmentation datasets (such as the COCO 2017 dataset) and paste these objects onto human images to simulate various degrees of occlusion scenes; generate abnormal interference images. Use OpenCV to simulate block distortion or other types of distortion effects. By summarizing the abnormal interference conditions in different scenarios, generate images such as block distortion and snowflake distortion. The prepared data will be divided into training sets, validation sets, and test sets to ensure the generalization ability of the model and the reliability of performance evaluation.
[0191] S302, model construction and training, using the ResNet-50 model under the PaddlePaddle deep learning framework, including feature extraction module, global average pooling module, fully connected layer and Softmax layer for binary classification training. The loss function uses the cross entropy loss function, and the optimization algorithm uses Adam. The specific process refers to S102.
[0192] S303, model testing and iterative update: Use the test set data to test the model effect. If the effect is not ideal, analyze the reasons, supplement the data and retrain the model until the model effect meets the expected requirements.
[0193] S400, build comprehensive quality evaluation module (see Figure 6 ). The comprehensive quality evaluation module consists of the following four sub-modules: human recognition module, objective quality analysis module, content quality analysis module, and weight module. Specifically, the workflow is as follows:
[0194] First, the input image is processed by the human recognition module. If there is a human target in the image, the image will enter the objective quality analysis module and the content quality analysis module for further analysis; if there is no human target, there is no need to perform the subsequent two module calculations.
[0195] In the objective quality analysis module, the image will generate the following four evaluation indicators: clarity (L), color distortion (C), brightness (B), and signal-to-noise ratio (S); the content quality analysis module classifies the image through the classification model and obtains the confidence of the following four categories: normal, occlusion, beyond the boundary, and abnormal distortion. We designed a comprehensive quality evaluation index Q, which ranges from 0 to 1. The closer to 0, the worse the image quality, and the closer to 1, the better the image quality. When there is no human target in the image, Q is 0. The calculation formula of Q combines the four evaluation indicators (L, C, B, S) of the above objective quality analysis module and the four category confidences of the content quality analysis module. The specific formula is as follows:
[0196]
[0197] Z=w L +w C +w B +w S +w N (P);
[0198] Where L max , C max , B max , S max are the maximum values of each index, P N The image obtained by content quality analysis is of normal confidence and is used for normalization processing.L 、w C 、w B 、w S 、w N is the weight coefficient of each indicator. Z is the normalization coefficient, which ensures that the value range of Q is between 0 and 1.
[0199] In the final decision-making stage, the comprehensive evaluation score Q is multiplied by the confidence level P of the output result of the artificial intelligence model. AI , and get the final decision:
[0200] D=Q·P AI (Q);
[0201] Among them, D is the final decision score, which is used for evaluation and decision-making.
[0202] To summarize, the comprehensive evaluation module aggregates the results of multi-dimensional objective quality analysis and content quality analysis, and combines them with the weight module for weighted calculation to finally obtain a comprehensive evaluation score Q. When the intelligent security inspection system encounters poor quality images, the system and method will give a lower image quality score Q through analysis. Multiplying it by the confidence level output by the artificial intelligence model will result in a lower value, thereby improving the robustness of the system when facing low-quality data.
[0203] The technical solution of the present invention carefully classifies the quality of complete screenshots of human bodies in video images through observation, induction and summary, and designs a complete quality evaluation system. A variety of specific image quality indicators and their mathematical expressions are set from the perspectives of objective quality and content quality. Finally, all indicators are summarized together to design an overall evaluation indicator Q and its mathematical expression.
[0204] The objective quality analysis module in the technical solution of the present invention is mainly implemented by analyzing the image in four dimensions of clarity, color distortion, illumination and noise through OpenCV; the content quality analysis module in this solution is implemented through the deep learning model obtained by the proposed training method.
[0205] The method proposed in the technical solution of the present invention can be applied to all intelligent monitoring systems deployed with artificial intelligence to improve the recognition accuracy and robustness of the system.
[0206] The technical solution of the present invention innovatively summarizes all low-quality image indicators to design a comprehensive quality indicator Q, and constructs a mathematical expression. This method can be deployed in any intelligent security video system to reduce the impact of low-quality images on the system by lowering the confidence level. For example, a low-quality image has a false alarm category confidence level I of 0.9 obtained by the intelligent algorithm model. It is identified as a low-quality image by this method, and the output quality indicator Q is 0.2. Q is multiplied by the confidence level I to obtain the final confidence level, which is reduced from 0.9 to 0.18. This confidence level will not be alarmed by the system, which improves the robustness of the system when facing low-quality images.
[0207] The specific implementation scheme of this embodiment can refer to the relevant description in the above embodiment, which will not be repeated here.
[0208] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.
[0209] It should be noted that, in the description of the present invention, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, the meaning of "plurality" refers to at least two.
[0210] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0211] The system, method, device, processor and readable storage medium thereof for evaluating and processing low-quality data for an intelligent safety monitoring system of the present invention are adopted. Aiming at the special needs of image quality assessment in the field of intelligent security, a multi-dimensional and multi-level analysis method is proposed to comprehensively examine the quality risks of input images, and the evaluation results are more accurate and reliable. A deep learning method is used to determine the existence of human targets and identify content-level quality defects, overcoming the limitation that traditional methods cannot understand image semantic information. Comprehensive quality scores are used to guide subsequent analysis modules to make reasonable responses, and the robustness of the system in the face of low-quality inputs is significantly improved without increasing computing overhead, thus avoiding erroneous analysis and saving invalid calculations. The evaluation modules and scoring strategies can be flexibly customized to adapt to different application scenarios, and have good versatility and practicality.
[0212] In this specification, the present invention has been described with reference to specific embodiments thereof. However, it is apparent that various modifications and variations may be made without departing from the spirit and scope of the present invention. Therefore, the specification and drawings should be regarded as illustrative rather than restrictive.
Claims
1. A system for evaluating and processing low-quality data for an intelligent safety monitoring system, characterized in that: The system comprises: Human body recognition module, used to identify whether the video image contains human targets: The objective quality analysis module is connected to the human body recognition module and is used to calculate and analyze the image quality indexes based on the image clarity, color, illumination, and noise angle using OpenCV to determine whether the image is of poor quality: The content quality analysis module is connected to the objective quality analysis module and is used to evaluate the image content.
2. The system for evaluating and processing low-quality data for an intelligent safety monitoring system according to claim 1, characterized in that: The system also includes a comprehensive evaluation module, which includes the human body recognition module, the objective quality analysis module and the content quality analysis module, and also includes a weight module. The input ends of the objective quality analysis module and the content quality analysis module are connected to the human body recognition module, and the output ends of the objective quality analysis module and the content quality analysis module are connected to the weight module; the comprehensive evaluation module is used to set the weight of each dimension according to different usage scenarios and comprehensively evaluate the image quality.
3. The system for evaluating and processing low-quality data for an intelligent safety monitoring system according to claim 1, characterized in that: The construction of the human body recognition module includes the following steps: Construct a dataset, obtain raw data from surveillance cameras in real scenes, use an open source human detection model to extract regions of interest containing human bodies as positive samples, and regions not containing human bodies as negative samples, and divide the data into training, validation, and test sets. Build and train the model, using the ResNet50 model under the PaddleCLS deep learning framework for binary classification training; Conduct model testing and iterative update and adjustment.
4. The system for evaluating and processing low-quality data for an intelligent safety monitoring system according to claim 1, characterized in that: The objective quality analysis module uses Laplace transform to detect the degree of image blur and analyzes the clarity of the image; the objective quality analysis module analyzes the color of the image by calculating the distribution and intensity evaluation of the colors in the image; the objective quality analysis module analyzes the illumination of the image by calculating the average brightness evaluation of the image; the objective quality analysis module analyzes the noise of the image by calculating the signal-to-noise ratio of the image.
5. The system for evaluating and processing low-quality data for an intelligent safety monitoring system according to claim 1, characterized in that: The content quality analysis module adopts a deep learning classification model to identify content-level quality issues in the input image that are closely related to intelligent analysis. By performing data enhancement operations on normal images, four types of data sets are constructed: normal images, out-of-boundary images, occluded images, and abnormally distorted images, and a multi-classification model based on the ResNet50 backbone network is trained.
6. The system for evaluating and processing low-quality data for an intelligent safety monitoring system according to claim 1, characterized in that: The construction of the content quality analysis module includes the following steps: Construct a data set to collect and divide four types of data into normal images, super-boundary images, occluded images, and abnormally distorted images: Use the ResNet50 model under the PaddleCLS deep learning framework for four-category training; Conduct model testing and iterative update and adjustment.
7. The system for evaluating and processing low-quality data for an intelligent safety monitoring system according to claim 1, characterized in that: The comprehensive evaluation module integrates the output results of the human recognition module, the objective quality analysis module and the content quality analysis module, and weightedly summarizes and normalizes them through the weight module.
8. A method for evaluating and processing low-quality data for an intelligent safety monitoring system based on the system of claim 1, characterized in that: The method comprises the following steps: (1) Construct a human recognition module to identify video images and determine whether they contain human targets: (2) The objective quality analysis module uses OpenCV to analyze the image quality indicators and determine whether it is a poor quality image: (3) Build a content quality analysis module to evaluate image content: (4) The comprehensive evaluation module sets the weight of each dimension according to different usage scenarios and comprehensively evaluates the image quality.
9. The method for implementing evaluation and processing of low-quality data for an intelligent safety monitoring system according to claim 8, characterized in that: The step (1) of constructing a human body recognition module to recognize a video image body comprises the following steps: (1.1) Construct a dataset, obtain raw data from surveillance cameras in real scenes, use an open source human detection model to extract regions of interest containing human bodies as positive samples, and regions not containing human bodies as negative samples, and divide the data into training set, validation set, and test set; (1.2) Build and train the model, using the ResNet50 model under the PaddleCLS deep learning framework for binary classification training; (1.3) Conduct model testing and iterative updates and adjustments.
10. The method for implementing evaluation and processing of low-quality data for an intelligent safety monitoring system according to claim 8, characterized in that: The step (2) specifically comprises the following steps: (2.1) Detect the blur level of the image through Laplace transform; (2.2) Evaluate whether the image has color distortion by calculating the distribution and intensity of colors in the image; (2.3) Evaluate the lighting conditions by calculating the average brightness of the image; (2.4) The noise in the image is evaluated by calculating the signal-to-noise ratio of the image.
11. The method for implementing evaluation and processing of low-quality data for an intelligent safety monitoring system according to claim 8, characterized in that: The step (3) of constructing the content quality analysis module specifically includes the following steps: (3.1) Construct a data set to collect and divide the data into four categories: normal images, out-of-boundary images, occluded images, and abnormally distorted images: (3.2) Use the ResNet50 model under the PaddleCLS deep learning framework to perform four-category training; (3.3) Conduct model testing and iterative update and adjustment.
12. The method for implementing evaluation and processing of low-quality data for an intelligent safety monitoring system according to claim 8, characterized in that: The step (4) specifically comprises the following steps: (4.1) Process the input image through the human body recognition module; (4.2) Determine whether there is a human target in the image. If yes, proceed to step (4.3); otherwise, proceed to step (4.4); (4.3) Analyzing the image through an objective quality analysis module and a content quality analysis module; (4.4) Calculate the comprehensive quality evaluation index Q; (4.5) Based on the comprehensive quality evaluation index Q and the confidence level P of the output results of the artificial intelligence model AI , and obtain the final decision.
13. A device for evaluating and processing low-quality data for an intelligent safety monitoring system, characterized in that: The device comprises: a processor configured to execute computer executable instructions; A memory storing one or more computer executable instructions, wherein when the computer executable instructions are executed by the processor, the steps of the method for evaluating and processing low-quality data for an intelligent safety monitoring system as described in any one of claims 8 to 12 are implemented.
14. A processor for implementing evaluation and processing of low-quality data for an intelligent safety monitoring system, characterized in that: The processor is configured to execute computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method for evaluating and processing low-quality data for an intelligent safety monitoring system as described in any one of claims 8 to 12 are implemented.
15. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program can be executed by a processor to implement the various steps of the method for evaluating and processing low-quality data for an intelligent safety monitoring system as described in any one of claims 8 to 12.
