Monitoring system based on intelligent image segmentation and recognition

Abstract

The invention relates to the technical field of computer vision and artificial intelligence, and discloses a monitoring system based on intelligent image segmentation and recognition, which is composed of an image acquisition module, an image preprocessing module, an intelligent segmentation and recognition module, a data analysis and decision module and an alarm and feedback module. According to the method, the noise in the image is removed by using methods such as Gaussian filtering, the smoothness of the image is improved, the influence of the noise on subsequent processing is reduced, the contrast and brightness of the image are improved through technologies such as contrast adjustment and histogram equalization, a target is clearer and more visible, the consistency of the image is kept under different environmental conditions, and the image quality is improved. The convolutional neural network is adopted for feature extraction and target classification, so that the accuracy of target recognition is remarkably improved, the accuracy of the system is further improved through continuous iteration and optimization, alarm is triggered only when an alarm module sets a hidden danger probability higher than 95%, and the beneficial effects of efficient target recognition and analysis and no false alarm are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and artificial intelligence, and in particular to a monitoring system based on intelligent image segmentation and recognition. Background Art

[0002] With the development of society, the demand for security monitoring and environmental monitoring is increasing. Traditional monitoring systems usually rely on manual monitoring or simple motion detection technology, which are often inefficient and prone to false alarms. In recent years, with the development of artificial intelligence technology, image recognition technology in computer vision has gained a large number of applications, such as text recognition in various product manuals, recognition of various abnormal objects in medical CT films, recognition of various satellite cloud images in weather forecasts, face recognition, recognition of key points on the human body, etc. Image recognition technology has been widely used in people's daily life and management decision-making.

[0003] For example, the publication number CN 114140738 A discloses an intelligent monitoring system for workshop safety hazards based on image recognition, including: a video acquisition module for acquiring parameter changes in the monitoring workshop, including high-definition cameras distributed at multiple locations in the monitoring workshop for acquiring images in the workshop; a network transmission module for transmitting the signal of the high-definition camera to the analysis system, and receiving the control command of the analysis system at the same time; an analysis system for importing the received images in the workshop into the detection model, analyzing and evaluating them through the computer vision target detection algorithm, identifying whether there are safety hazards in the current workshop, and identifying whether there are safety hazards in the current workshop; an alarm module for checking whether to send alarm information to the outside according to the results of the analysis system. The present invention combines the computer vision target detection algorithm, takes into account both real-time and accuracy, improves monitoring efficiency through intelligent and automated monitoring and calculation, and can also save manpower and material costs and inspection time.

[0004] Although the aforementioned technical solution analyzes safety hazards through image recognition, when the environment in the workshop changes, such as light changes, object movement, and shadows, the aforementioned technical solution will interfere with the normal operation of the image recognition algorithm, thereby causing false alarms or missed alarms. Summary of the invention

[0005] Technical issues solved:

[0006] In view of the deficiencies of the prior art, the present invention provides a monitoring system based on intelligent image segmentation and recognition, which has the advantages of efficient recognition and analysis of targets without false alarms, and solves the problems of the above-mentioned technologies.

[0007] Technical solution:

[0008] To achieve the above object, the present invention provides the following technical solutions: a monitoring system based on intelligent image segmentation and recognition, the monitoring system is composed of an image acquisition module, an image preprocessing module, an intelligent segmentation and recognition module, a data analysis and decision module and an alarm and feedback module;

[0009] The image acquisition module uses a high-definition camera to collect images, fully covering the workshop area;

[0010] The image preprocessing module implements Gaussian filtering to remove noise, uses contrast adjustment or histogram equalization to perform image enhancement to improve image quality and visibility, and adds image standardization processing to ensure that images under different time and environmental conditions are consistent;

[0011] The intelligent segmentation and recognition module uses convolutional neural networks for feature extraction and target classification to improve the accuracy of target recognition, uses U-Net for semantic segmentation to accurately extract target areas, uses Mask R-CNN for instance segmentation to locate and segment single targets, implements a panoramic segmentation algorithm to handle multi-target location and segmentation in complex scenes, and uses a deep learning tracker to track targets in consecutive frames to reduce false positives caused by target movement;

[0012] The data analysis and decision module comprehensively analyzes the segmentation and recognition results, applies multi-level data analysis algorithms to reduce false positives, and introduces anomaly detection algorithms to detect irregular patterns and potential hidden dangers in images;

[0013] The alarm and feedback module is used to set the alarm triggering conditions according to the analysis results and the output of the decision module, ensuring that the alarm is triggered only when there is a high probability of ≥95% of hidden dangers;

[0014] The Gaussian filter noise removal expression in the image preprocessing module is:

[0015]

[0016] Where: I in(x′,y′) is the pixel value of the input image at position (x′, y′); I out(x,y) is the pixel value of the filtered image at position (x, y), σ is the standard deviation of the Gaussian kernel, which determines the smoothness of the filter;

[0017] The formula smoothes the image by calculating a weighted average, where the weight is determined by a Gaussian function, and the farther away the pixel, the smaller the weight;

[0018] The image standardization processing expression is:

[0019]

[0020] Where: Iin(x,y) is the pixel value of the input image at position (x, y); I out(x,y) is the pixel value of the standardized image at position (x, y); μ is the mean of the image; σ is the standard deviation of the image; the image is standardized to have zero mean and unit standard deviation.

[0021] Preferably, the contrast adjustment expression in the image preprocessing module is:

[0022] I out(x,y) =α*I in(x,y) +β

[0023] Where: I in(x,y) is the pixel value of the input image at position (x, y); I out(x,y) is the pixel value of the adjusted image at position (x, y); α is the contrast gain factor; β is the brightness offset;

[0024] The contrast adjustment expression adjusts the contrast and brightness of the image through linear transformation, α controls the contrast, and β controls the brightness.

[0025] Preferably, the histogram equalization expression in the image preprocessing module is:

[0026] Cumulative Distribution Function (CDF):

[0027]

[0028] The pixel value after equalization:

[0029] I out(x,y) =round(255*CDF(I in(x,y) ))

[0030] Where: p(j) is the probability of pixel value j; N is the total number of pixels in the image; CDF(i) is the cumulative distribution function of pixel values ​​less than or equal to i; I out(x,y) is the pixel value of the equalized image at position (x, y);

[0031] The histogram equalization expression enhances the contrast of the image by adjusting the distribution of pixel values.

[0032] Preferably, the feature extraction and target classification expressions in the intelligent segmentation and recognition module are:

[0033] F=CNN(I)

[0034] Where: I is the input image; CNN is the convolutional neural network; F is the extracted feature.

[0035] Preferably, the semantic segmentation expression in the intelligent segmentation and recognition module is: S=U-Net(I); wherein: I is the input image; U-Net is the U-Net model; S is the semantic segmentation result, marking the target area in the image;

[0036] The instance segmentation expression in the intelligent segmentation and recognition module is: M, B = Mask-RCNN (I); where: I is the input image; Mask-RCNN is the Mask-RCNN model; M is the segmentation mask, marking the area of ​​each object; B is the bounding box, locating the position of each object;

[0037] The panoramic segmentation expression in the intelligent segmentation and recognition module is: P = PanopticSegmentation(I); wherein: I is the input image; PanopticSegmentation is the panoramic segmentation algorithm; P is the panoramic segmentation result, including the positioning and segmentation of multiple targets.

[0038] Preferably, the target tracking expression in the intelligent segmentation and recognition module is:

[0039] T=DeepTrachker(F,F prev )

[0040] Where: F is the feature in the current frame; F prev is the feature in the previous frame; DeepTrachker is the deep learning tracker; T is the target tracking result, which reduces the false positives caused by target movement.

[0041] Preferably, the comprehensive analysis, segmentation and recognition result expression in the data analysis and decision module is:

[0042] A(x,y)=w s *S(x,y)+w m *M(x,y)+w b *B(x,y)+w p *P(x,y)+w t *T(x,y)

[0043] Where: S(x,y) is the value of the semantic segmentation result at position (x,y); M(x,y) is the value of the instance segmentation mask at position (x,y); B(x,y) is the value of the bounding box information at position (x,y); P(x,y) is the value of the panoramic segmentation result at position (x,y); T(x,y) is the value of the target tracking result at position (x,y); w s ,w m ,w b ,w p ,w tis the weight coefficient of the corresponding result; A(x,y) is the result after comprehensive analysis.

[0044] Preferably, the expression for reducing false alarms by applying a multi-level data analysis algorithm in the data analysis and decision-making module is: R(x,y)=f(A(x,y)); wherein: A(x,y) is the result after comprehensive analysis; f is a multi-level data analysis algorithm function; and R(x,y) is the result after reducing false alarms.

[0045] Preferably, the anomaly detection algorithm is expressed as:

[0046] D(x,y)=y(R(x,y))

[0047] Where: D(x,y) is the result after reducing false alarms; g is the anomaly detection algorithm function; R(x,y) is the detected anomaly or potential hidden danger.

[0048] Preferably, the alarm and feedback module expression is:

[0049]

[0050] Where: P is the probability of hidden danger, which is 0.95 here; Alarm is the alarm status, 1 means triggering the alarm, and 0 means not triggering the alarm.

[0051] Compared with the prior art, the present invention provides a monitoring system based on intelligent image segmentation and recognition, which has the following beneficial effects:

[0052] 1. The present invention uses Gaussian filtering and other methods to remove noise from the image, improve the smoothness of the image, reduce the impact of noise on subsequent processing, and improve the contrast and brightness of the image through contrast adjustment and histogram equalization. The technology makes the target more visible and maintains the consistency of the image under different environmental conditions. The convolutional neural network is used for feature extraction and target classification, thereby significantly improving the accuracy of target recognition. By using U-Net for semantic segmentation, the target area is accurately extracted; Mask is used R-CNN performs instance segmentation, can accurately locate and divide each target, and reduce the possibility of false alarms. It applies panoramic segmentation technology to handle multiple targets in complex scenes, ensuring that the system can work effectively in complex environments. It uses deep learning trackers to track targets in consecutive frames, which can track the movement of targets and reduce false alarms caused by target motion. It conducts a comprehensive analysis of the results of segmentation and recognition, and combines the outputs of different data sources to improve the stability and accuracy of the results. It applies multi-level data analysis algorithms to reduce the occurrence of false alarms through continuous iteration and optimization. It introduces anomaly detection algorithms to identify irregular patterns and potential hidden dangers in images, further improving the accuracy of the system. The alarm module sets the probability of hidden dangers above 95% to trigger the alarm, achieving the beneficial effect of efficient recognition and analysis of targets without false alarms.

[0053] 2. The present invention uses Gaussian filtering and other technologies to remove noise from images, improve image clarity, reduce analysis errors caused by image quality problems, use contrast adjustment and histogram equalization technology to enhance the contrast and brightness of images, make targets and abnormal phenomena more prominent, and facilitate subsequent analysis and processing. Convolutional neural networks are used for feature extraction and target classification, so that the system can identify key elements in the image and accurately classify them. This helps to accurately identify different objects and phenomena in security analysis, and semantic segmentation is performed through U-Net to accurately extract target areas; instance segmentation is performed through Mask R-CNN to accurately locate and segment each target, ensuring clear identification of different targets, providing detailed target information for security analysis, and achieving the beneficial effect of promoting security analysis and prediction of related scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a schematic diagram of the system of the present invention; DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] See also Figure 1 , a monitoring system based on intelligent image segmentation and recognition, the monitoring system consists of an image acquisition module, an image preprocessing module, an intelligent segmentation and recognition module, a data analysis and decision module, and an alarm and feedback module;

[0057] The image acquisition module uses a high-definition camera to collect images, fully covering the workshop area;

[0058] The image preprocessing module implements Gaussian filtering to remove noise, uses contrast adjustment or histogram equalization to perform image enhancement to improve image quality and visibility, and adds image standardization processing to ensure that images under different time and environmental conditions are consistent;

[0059] The intelligent segmentation and recognition module uses convolutional neural networks for feature extraction and target classification to improve the accuracy of target recognition, uses U-Net for semantic segmentation to accurately extract target areas, uses Mask R-CNN for instance segmentation to locate and segment single targets, implements a panoramic segmentation algorithm to handle multi-target location and segmentation in complex scenes, and uses a deep learning tracker to track targets in consecutive frames to reduce false positives caused by target movement;

[0060] The data analysis and decision module comprehensively analyzes the segmentation and recognition results, applies multi-level data analysis algorithms to reduce false positives, and introduces anomaly detection algorithms to detect irregular patterns and potential hidden dangers in images;

[0061] The alarm and feedback module is used to set the alarm triggering conditions according to the analysis results and the output of the decision module, ensuring that the alarm is triggered only when there is a high probability of ≥95% of hidden dangers;

[0062] The Gaussian filter noise removal expression in the image preprocessing module is:

[0063]

[0064] Where: I in(x′,y′) is the pixel value of the input image at position (x′, y′); I out(x,y) is the pixel value of the filtered image at position (x, y), σ is the standard deviation of the Gaussian kernel, which determines the smoothness of the filter;

[0065] The formula smoothes the image by calculating a weighted average, where the weight is determined by a Gaussian function, and the farther away the pixel, the smaller the weight;

[0066] The image standardization processing expression is:

[0067]

[0068] Where: I in(x,y) is the pixel value of the input image at position (x, y); I out(x,y) is the pixel value of the standardized image at position (x, y); μ is the mean of the image; σ is the standard deviation of the image; the image is standardized to have zero mean and unit standard deviation.

[0069] Use of high-definition cameras: The system is equipped with high-definition cameras to ensure full coverage of the workshop area and obtain high-quality images, providing a clear data basis for subsequent processing.

[0070] Image denoising: Use methods such as Gaussian filtering to remove noise from images, improve the smoothness of the image, and reduce the impact of noise on subsequent processing.

[0071] Image enhancement: Improve the contrast and brightness of the image through contrast adjustment and histogram equalization, making the target more visible. This helps to maintain image consistency under different environmental conditions.

[0072] Feature extraction and classification: Convolutional neural networks are used for feature extraction and target classification, which significantly improves the accuracy of target recognition.

[0073] Semantic segmentation and instance segmentation: By using U-Net for semantic segmentation, the target area can be accurately extracted; using Mask R-CNN for instance segmentation, each target can be accurately located and divided, reducing the possibility of false positives.

[0074] Multi-target segmentation: Apply panoramic segmentation technology to handle multiple targets in complex scenes and ensure that the system can work effectively in complex environments.

[0075] Target tracking: Using deep learning trackers to track targets in consecutive frames can track the movement of targets and reduce false positives caused by target motion.

[0076] Comprehensive result analysis: Comprehensive analysis of the results of segmentation and recognition is performed, combining the outputs of different data sources to improve the stability and accuracy of the results.

[0077] Multi-level data analysis: Apply multi-level data analysis algorithms to reduce the occurrence of false positives through continuous iteration and optimization.

[0078] Anomaly Detection: The introduction of anomaly detection algorithms can identify irregular patterns and potential hazards in images, further improving the accuracy of the system.

[0079] HD image acquisition: HD cameras are used for comprehensive image acquisition to ensure coverage of the entire workshop area. This provides high-resolution and detailed image data for safety analysis, ensuring that potential safety hazards can be accurately captured and identified.

[0080] Noise removal: Remove noise from the image through Gaussian filtering and other techniques to improve image clarity and reduce analysis errors caused by image quality issues.

[0081] Image enhancement: Use contrast adjustment and histogram equalization techniques to enhance the contrast and brightness of the image, making the target and anomalies more prominent and facilitating subsequent analysis and processing.

[0082] Feature extraction and classification: Using convolutional neural networks for feature extraction and target classification enables the system to identify key elements in an image and accurately classify them. This helps to accurately identify different objects and phenomena in security analysis.

[0083] Semantic segmentation and instance segmentation: U-Net is used for semantic segmentation to accurately extract the target area; Mask R-CNN is used for instance segmentation to accurately locate and segment each target. These technologies ensure clear identification of different targets and provide detailed target information for security analysis.

[0084] Multi-target segmentation and tracking: Use panoptic segmentation technology to process multiple targets in complex scenes, and track the targets through deep learning trackers to ensure continuous monitoring of the target status even in dynamic scenes. This helps to detect and predict potential safety issues in a timely manner.

[0085] Comprehensive result analysis: Comprehensive analysis of the results of segmentation and recognition, combining the results of multiple data sources such as semantic segmentation, instance segmentation, bounding box information, panoptic segmentation, and object tracking. This comprehensive analysis provides a more comprehensive perspective and helps to accurately identify and predict safety hazards.

[0086] Multi-level data analysis algorithm: Through multi-level data analysis algorithms, the results are optimized, false positives are reduced, and the system's ability to identify abnormal situations is improved. This helps to analyze potential risks more accurately and provide more reliable predictions.

[0087] Anomaly detection algorithm: The introduction of anomaly detection algorithm can identify irregular patterns and potential hidden dangers in images. This technology helps the system foresee possible security issues, issue early warnings, and reduce the occurrence of security incidents.

[0088] Intelligent alarm triggering: Set alarm triggering conditions to trigger the alarm only when the probability of hidden dangers reaches a certain threshold. This design reduces the possibility of false alarms while ensuring that the system can issue an alarm in time when there is a real safety risk.

[0089] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A monitoring system based on intelligent image segmentation and recognition, characterized in that: The monitoring system is composed of an image acquisition module, an image preprocessing module, an intelligent segmentation and recognition module, a data analysis and decision module, and an alarm and feedback module; The image acquisition module uses a high-definition camera to collect images, fully covering the workshop area; The image preprocessing module implements Gaussian filtering to remove noise, uses contrast adjustment or histogram equalization to perform image enhancement to improve image quality and visibility, and adds image standardization processing to ensure that images under different time and environmental conditions are consistent; The intelligent segmentation and recognition module uses convolutional neural networks for feature extraction and target classification to improve the accuracy of target recognition, uses U-Net for semantic segmentation to accurately extract target areas, uses Mask R-CNN for instance segmentation to locate and segment single targets, implements a panoramic segmentation algorithm to handle multi-target location and segmentation in complex scenes, and uses a deep learning tracker to track targets in consecutive frames to reduce false positives caused by target movement; The data analysis and decision module comprehensively analyzes the segmentation and recognition results, applies multi-level data analysis algorithms to reduce false positives, and introduces anomaly detection algorithms to detect irregular patterns and potential hidden dangers in images; The alarm and feedback module is used to set the alarm triggering conditions according to the analysis results and the output of the decision module, ensuring that the alarm is triggered only when there is a high probability of ≥95% of hidden dangers; The Gaussian filter noise removal expression in the image preprocessing module is: Where: I in (x′, y′) is the pixel value of the input image at position (x′, y′); I out(x,y) is the pixel value of the filtered image at position (x, y), σ is the standard deviation of the Gaussian kernel, which determines the smoothness of the filter; The formula smoothes the image by calculating a weighted average, where the weight is determined by a Gaussian function, and the farther away the pixel, the smaller the weight; The image standardization processing expression is: Where: I in(x,y) is the pixel value of the input image at position (x, y); I out(x,y) is the pixel value of the standardized image at position (x, y); μ is the mean of the image; σ is the standard deviation of the image; the image is standardized to have zero mean and unit standard deviation.

2. The monitoring system based on intelligent image segmentation and recognition according to claim 1, characterized in that: The contrast adjustment expression in the image preprocessing module is: I out(x,y) =α*I in(x,y) +b Where: I in(x,y) is the pixel value of the input image at position (x, y); I out(x,y) is the pixel value of the adjusted image at position (x, y); α is the contrast gain factor; β is the brightness offset; The contrast adjustment expression adjusts the contrast and brightness of the image through linear transformation, α controls the contrast, and β controls the brightness.

3. The monitoring system based on intelligent image segmentation and recognition according to claim 1, characterized in that: The histogram equalization expression in the image preprocessing module is: Cumulative Distribution Function (CDF): The pixel value after equalization: I out(x,y) =round(255*CDF(I in(x,y) )) Where: p(j) is the probability of pixel value j; N is the total number of pixels in the image; CDF(i) is the cumulative distribution function of pixel values ​​less than or equal to i; I out(x,y) is the pixel value of the equalized image at position (x, y); The histogram equalization expression enhances the contrast of the image by adjusting the distribution of pixel values.

4. The monitoring system based on intelligent image segmentation and recognition according to claim 2 or 3, characterized in that: The feature extraction and target classification expressions in the intelligent segmentation and recognition module are as follows: F=CNN(I) Where: I is the input image; CNN is the convolutional neural network; F is the extracted feature.

5. The monitoring system based on intelligent image segmentation and recognition according to claim 4 is characterized in that: The semantic segmentation expression in the intelligent segmentation and recognition module is: S = U-Net (I); where: I is the input image; U-Net is the U-Net model; S is the semantic segmentation result, marking the target area in the image; The instance segmentation expression in the intelligent segmentation and recognition module is: M, B = Mask-RCNN (I); where: I is the input image; Mask-RCNN is the Mask-RCNN model; M is the segmentation mask, marking the area of ​​each object; B is the bounding box, locating the position of each object; The panoramic segmentation expression in the intelligent segmentation and recognition module is: P = PanopticSegmentation(I); wherein: I is the input image; PanopticSegmentation is the panoramic segmentation algorithm; P is the panoramic segmentation result, including the positioning and segmentation of multiple targets.

6. The monitoring system based on intelligent image segmentation and recognition according to claim 5, characterized in that: The target tracking expression in the intelligent segmentation and recognition module is: T=DeepTracker(F,F prev ) Where: F is the feature in the current frame; F prev is the feature in the previous frame; DeepTracker is the deep learning tracker; T is the target tracking result, which reduces the false positives caused by target movement.

7. The monitoring system based on intelligent image segmentation and recognition according to claim 6, characterized in that: The expression of the comprehensive analysis, segmentation and recognition results in the data analysis and decision module is: A(x,y)=w s *S(x,y)+w m *M(x,y)+w b *B(x,y)+w p *P(x,y)+w t *T(x,y) Where: S(x,y) is the value of the semantic segmentation result at position (x,y); M(x,y) is the value of the instance segmentation mask at position (x,y); B(x,y) is the value of the bounding box information at position (x,y); P(x,y) is the value of the panoramic segmentation result at position (x,y); T(x,y) is the value of the target tracking result at position (x,y); w s ,w m ,w b ,w p ,w t is the weight coefficient of the corresponding result; A(x,y) is the result after comprehensive analysis.

8. The monitoring system based on intelligent image segmentation and recognition according to claim 7, characterized in that: The expression for reducing false alarms by applying a multi-level data analysis algorithm in the data analysis and decision-making module is: R(x,y)=f(A(x,y)); wherein: A(x,y) is the result after comprehensive analysis; f is the multi-level data analysis algorithm function; and R(x,y) is the result after reducing false alarms.

9. The monitoring system based on intelligent image segmentation and recognition according to claim 8, characterized in that: The anomaly detection algorithm expression is: D(x,y)=y(R(x,y)) Where: D(x,y) is the result after reducing false alarms; g is the anomaly detection algorithm function; R(x,y) is the detected anomaly or potential hidden danger.

10. The monitoring system based on intelligent image segmentation and recognition according to claim 8, characterized in that: The alarm and feedback module expression is: Where: P is the probability of hidden danger, which is 0.95 here; Alarm is the alarm status, 1 means triggering the alarm, and 0 means not triggering the alarm.

Images