Air valve detection and identification method based on deep learning

Through the deep learning-based air valve detection method, the yolox object detection model and binary classification model are used to solve the problems of traditional sensor reduction and insufficient real-time monitoring capabilities, and high-precision air valve status detection and opening degree quantification, improving the system The stability and adaptability of the company reduces maintenance requirements and costs.

CN119130979APending Publication Date: 2024-12-13ZHEJIANG SINOKING AIR CONDITIONING & REFRIGERATION CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411251241.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-07
Publication Date
2024-12-13

AI Technical Summary

Technical Problem

In the existing air valve state detection system, traditional mechanical and electronic sensors are susceptible to physical wear and environmental interference, resulting in reduced accuracy and difficulty in providing continuous quantization data, limiting control fineness, and data processing delays, affecting real-time monitoring. ability.

Method used

Using deep learning-based air valve detection and recognition methods, using yolox object detection model and binary classification model, the monitoring accuracy is improved through machine learning algorithms, reduce misjudgment, and accurately detect the switch status and opening degree of the air valve, reducing physical sensors. dependency.

Benefits of technology

It improves monitoring accuracy and real-time response capabilities, reduces misjudgment, optimizes image processing flow, improves system stability and adaptability, supports remote monitoring, fault self-diagnosis and predictive maintenance, and reduces maintenance requirements and costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119130979A_ABST
    Figure CN119130979A_ABST
Patent Text Reader

Abstract

The invention relates to the field of air valve detection, in particular to an air valve detection and recognition method based on deep learning, and the method comprises the following steps: S1, obtaining an air valve image; s2, detecting the air valve image by using a yolox target detection model to obtain the type and position information of the air valve in the image; s3, the obtained position information of the air valve is transmitted into the trained binary classification model residual network ResNet50 to obtain the on-off state of the air valve; s4, performing image preprocessing on the open air valve to obtain edge feature information of air valve blades, finding out the contour with the largest area in all external contours through contour detection, calculating the area of a black area, and calculating the area of the black area according to the predefined minimum area in the closed state of the air valve and the maximum area in the open state of the air valve; estimating the opening and closing degree of the air valve; the monitoring precision is improved and misjudgment is reduced through a machine learning algorithm by adopting a yolox target detection model and a dichotomy model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of air valve detection, and more specifically to an air valve detection and identification method based on deep learning. Background Art

[0002] In modern damper status detection systems, mechanical and electronic sensors are often used in combination to identify the switch status of the damper and quantify its degree of opening. Due to the limited functionality of a single sensor, in order to obtain more accurate and comprehensive monitoring data, multiple sensors are often used in practice to work together; traditional mechanical sensors can convert the position change of the damper into a relative change in resistance value, thereby generating a power signal by integrating mechanical components such as potentiometers. The main advantages of this method are its simple design and low cost. However, due to their reliance on physical contact, these sensors are easily affected by continuous operation and environmental factors, resulting in a short service life and reduced monitoring performance over time;

[0003] Traditional electronic sensors, such as Hall-effect based devices, accurately determine the position of the damper by sensing changes in the magnetic field. The main advantage of electronic sensors over mechanical sensors is their excellent stability and long service life, mainly because they do not require physical contact, thus reducing the risk of wear and failure. In addition, advanced electronic flow sensors can not only provide binary information on the on / off state of the damper, but also continuously quantify the degree of opening of the damper, which makes them also excellent in complex applications that require fine control;

[0004] However, whether it is a mechanical sensor or an electronic sensor, when monitoring the status of the air valve, the accuracy may decrease due to physical wear or environmental interference; and it is difficult to provide continuous quantitative data on the status of the air valve, limiting the precision of control; there may also be data processing delays, affecting real-time monitoring capabilities; and poor installation environment affecting the stability of the sensor. Summary of the invention

[0005] The purpose of the present invention is to provide a wind valve detection and identification method based on deep learning, which no longer relies on traditional sensors to detect wind valves. It adopts the yolox target detection model and the binary classification model, and improves the monitoring accuracy and reduces misjudgment through machine learning algorithms.

[0006] The purpose of the present invention is achieved through the following technical solutions:

[0007] A method for detecting and identifying an air valve based on deep learning, the method comprising the following steps:

[0008] Step S1: Acquire the air valve image;

[0009] Step S2: Use the yolox target detection model to detect the air valve image and obtain the type and position information of the air valve in the image;

[0010] Step S3: The acquired position information of the air valve is passed into the trained binary classification model residual network ResNet50 to obtain the switch state of the air valve;

[0011] Step S4: Preprocess the image of the air valve of the open type to obtain the edge feature information of the air valve blade, find the contour with the largest area among all external contours through contour detection, calculate the area of ​​the black area, and estimate the opening and closing degree of the air valve according to the pre-defined minimum area when the air valve is closed and the maximum area when the air valve is open;

[0012] In S1, a picture of the air valve video to be analyzed is taken at intervals of five frames as an air valve image;

[0013] Before detection, it is necessary to obtain the weight of the air valve, that is, to perform annotation training on the air valve data; annotate the air valve category on the collected original image, integrate the original image and the annotated data into an air valve detection data set, and train the Yolox target detection model, which specifically includes the following steps:

[0014] Step S21: Collecting initial data: Using a fixed industrial camera to collect images of air valves in different states, at different times and at different locations to form an air valve image data set;

[0015] Step S22: label each sub-image in the air valve image data set using LabelImg software, record the location and category information of the air valve, and save it in txt text format;

[0016] Step S23: randomly dividing the labeled air valve image dataset into a training set and a test set in a ratio of 4:1, wherein the training set includes 801 images and the test set includes 198 images;

[0017] Step S24: adopting the transfer learning strategy to train the Yolox target detection model;

[0018] In step S24, the entire training process includes 300 epochs in total. In the first 50 epochs, a frozen training phase is implemented. In this phase, only the neck network Neck and the head network Head of the model are fine-tuned, while the backbone network Backdone is kept unchanged. In the frozen phase, the batch size is set to 16; the next 250 epochs are the unfreezing training phase, in which the entire network structure participates in the training, and the batch size is set to 8;

[0019] The step S3 specifically comprises the following steps:

[0020] Step S31: using 500 open air valve state diagrams and closed air valve state diagrams as a data set for the binary classification model residual network ResNet50, wherein the data set is divided into a training set and a test set in a 4:1 ratio, and the learning rate is set to 0.00001 during training, the batch size is 32, and the training is performed for 100 rounds;

[0021] Step S31: classify the air valve position information obtained by the yolox target detection model to obtain the switch state and air valve position information of the air valve;

[0022] The image preprocessing in step S4 is as follows: converting the image into a grayscale image by grayscale conversion, retaining the contour and texture information of the image, reducing the image noise by applying a filter, and using Otsu's method to determine the optimal threshold based on the histogram statistical analysis of the image, thereby converting the image into a binary image. The binary processing will enhance the contrast between the leaf and the background, so that the leaf area becomes black and the background becomes white;

[0023] The formula of the Otsu method is Formula 1:

[0024] If the current threshold is x, where ω0 is the proportion of foreground points, ω1 = 1-ω0 is the proportion of background points, μ0 = foreground grayscale mean, μ1 = background grayscale mean; u = ω0*μ0+ω1*μ1 global grayscale mean:

[0025] y=μ0(μ0-u)*(μ0-u)+ω1(μ1-u)*(μ1-u) Equation 1

[0026] After simplification, we get formula 2:

[0027] y=ω0*(1-ω0)*(μ0-μ1)*(μ0-μ1) Equation 2

[0028] The objective function is y, the larger y is, the better the threshold value of x is;

[0029] In step S4, edge feature information of the air valve blade is obtained by using Canny edge detection, and the contour with the largest area among all external contours is found by using contour detection;

[0030] In the step S4: the picture of the air valve fully open and the picture of the air valve fully closed are input into the algorithm to calculate the area of ​​the black area, which are recorded as A_open and A_closed respectively;

[0031] For the damper image in any state, calculate the black area A_current, and then use linear interpolation or other mathematical models to quantify the degree of opening and closing of the blade Valve_Opening_Degree;

[0032] The formula is shown in formula 3:

[0033]

[0034] In step S4, a single air valve in each image is analyzed to determine its opening degree in the single image, and outliers are identified and excluded by comprehensive statistical methods, and the average value of the remaining data is calculated using a robust statistical method, thereby obtaining an accurate and reliable opening degree of each air valve.

[0035] The beneficial effects of the present invention are:

[0036] The yolox target detection model and the binary classification model are used to improve the monitoring accuracy and reduce misjudgment through machine learning algorithms; the air valve opening is further analyzed according to the detection results in the binary classification, and the quantitative evaluation of the air valve opening degree is achieved through image processing related algorithms; the algorithm processing flow is optimized to achieve fast image processing and state judgment, and improve the real-time response capability of the system; the yolox target detection model used takes into account a variety of environmental factors during the training process, which improves the stability and adaptability of the system in different environments; it integrates advanced deep learning and image processing technologies to support remote monitoring, fault self-diagnosis and predictive maintenance; it reduces dependence on physical sensors and reduces maintenance requirements and costs through algorithm optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0038] Figure 1 It is a schematic diagram of the air valve detection and identification method based on deep learning of the present invention. DETAILED DESCRIPTION

[0039] The present invention is further described in detail below with reference to the accompanying drawings.

[0040] like Figure 1 As shown, the steps and functions of a deep learning-based air valve detection and identification method are described in detail below;

[0041] A method for detecting and identifying an air valve based on deep learning, the method comprising the following steps:

[0042] Step S1: obtaining an initial image; obtaining an air valve image;

[0043] In S1, a picture of the air valve video to be analyzed is taken at intervals of five frames as an air valve image;

[0044] Step S2: Image target detection: Use the yolox target detection model to detect the air valve image and obtain the type and position information of the air valve in the image;

[0045] Before detection, it is necessary to obtain the weight of the air valve, that is, to perform annotation training on the air valve data; annotate the air valve category on the collected original image, integrate the original image and the annotated data into an air valve detection data set, and train the Yolox target detection model, which specifically includes the following steps:

[0046] Step S21: Collecting initial data: Using a fixed industrial camera to collect images of air valves in different states, at different times and at different locations to form an air valve image data set;

[0047] Step S22: label each sub-image in the air valve image data set using LabelImg software, record the location and category information of the air valve, and save it in txt text format;

[0048] Step S23: randomly dividing the labeled air valve image dataset into a training set and a test set in a ratio of 4:1, wherein the training set includes 801 images and the test set includes 198 images;

[0049] Step S24: adopting the transfer learning strategy to train the Yolox target detection model;

[0050] In step S24, the entire training process includes 300 epochs in total. In the first 50 epochs, a frozen training phase is implemented. In this phase, only the neck network Neck and the head network Head of the model are fine-tuned, while the backbone network Backdone is kept unchanged. In the frozen phase, the batch size is set to 16 to more effectively utilize computing resources. The next 250 epochs are unfreezing training phases, and the entire network structure participates in the training to refine and optimize the parameters of all network layers. Since the unfreezing phase involves more parameter updates, the batch size is set to 8 to adapt to the limitation of video memory capacity and ensure the stability of the training process.

[0051] Step S3: image classification detection; the acquired position information of the air valve is passed into the trained binary classification model residual network ResNet50 to obtain the switch state of the air valve;

[0052] The step S3 specifically comprises the following steps:

[0053] Step S31: using 500 open air valve state diagrams and closed air valve state diagrams as a data set for the binary classification model residual network ResNet50, wherein the data set is divided into a training set and a test set in a 4:1 ratio, and the learning rate is set to 0.00001 during training, the batch size is 32, and the training is performed for 100 rounds;

[0054] Step S31: classify the air valve position information obtained by the yolox target detection model to obtain the switch state and air valve position information of the air valve;

[0055] Step S4: Image processing to quantify the degree of opening; perform image preprocessing on the air valve of type open to obtain the edge feature information of the air valve blades, and then find the contour with the largest area among all external contours through contour detection, which represents the external contour information of all blades of the air valve. Finally, calculate the area of ​​the black area, and estimate the degree of opening and closing of the air valve based on the predefined minimum area (the state of the air valve closed) and maximum area (the state of the air valve open). The specific operations are as follows:

[0056] Step S41: Image preprocessing:

[0057] The image is converted into a grayscale image by grayscale conversion, the contour and texture information of the image are retained, the image noise is reduced by applying a filter, and the Otsu method is used to determine the optimal threshold based on the histogram statistical analysis of the image, so as to convert the image into a binary image. The binary processing will enhance the contrast between the leaf and the background, making the leaf area black and the background white.

[0058] Grayscale conversion is the process of converting a color image into a grayscale image. Color images usually contain three color channels: red, green, and blue, while grayscale images only have one channel. Grayscale images have a smaller amount of data and are more efficient to process, while retaining the image's contour and texture information;

[0059] Filtering denoising is the process of reducing image noise by applying filters. Noise can be caused by a variety of factors, such as thermal noise of the camera sensor, changes in lighting conditions, etc. Gaussian filter is a commonly used linear filter that reduces noise by smoothing the image while preserving the edge information of the image as much as possible;

[0060] The formula of the Otsu method is as follows: The Otsu method is an automatic threshold selection method, which determines the optimal threshold based on the histogram statistical analysis of the image, thereby converting the image into a binary image. The binarization process will enhance the contrast between the leaf and the background, so that the leaf area becomes black and the background becomes white;

[0061] If the current threshold is x, where ω0 is the proportion of foreground points, ω1 = 1-ω0 is the proportion of background points, μ0 = foreground grayscale mean, μ1 = background grayscale mean; u = ω0*μ0+ω1*μ1 global grayscale mean:

[0062] y=μ0(μ0-u)*(μ0-u)+ω1(μ1-u)*(μ1-u) Equation 1

[0063] After simplification, we get formula 2:

[0064] y=ω0*(1-ω0)*(μ0-μ1)*(μ0-μ1) Equation 2

[0065] The objective function is y, the larger y is, the better the threshold value of x is;

[0066] Step S42: Extract the air valve contour:

[0067] The edge feature information of the air valve blade is obtained through Canny edge detection, and the contour with the largest area among all external contours is found through contour detection;

[0068] In binary image processing, the maximum connected block refers to the largest area composed of adjacent pixels that are connected to each other and have the same grayscale or color value. In specific application scenarios, such as analyzing the opening state of a damper, the multiple blades and the outermost metal frame of the damper (if they have the same color) can be regarded as a whole maximum connected block. In contrast, those areas that do not belong to this maximum connected block are considered to be the areas exposed when the blades are opened;

[0069] Edge detection is to identify the edge by detecting low-level mutations in features such as brightness and color in the image. The edge of an image refers to the part of the local area of ​​the image where the brightness changes significantly. The grayscale profile of this area can be regarded as a step, that is, a sharp change from one grayscale value in a small buffer area to another grayscale value with a large grayscale difference. The edge of the image concentrates most of the information of the image, so the determination and extraction of the edge of the image is very important for the recognition and understanding of the entire image scene, and it is also an important feature that image segmentation relies on.

[0070] Canny edge detection is a method of detecting edges using a multi-level edge detection algorithm, which aims to significantly reduce the data size of the image while retaining the original image properties. The main steps of the Canny algorithm are to use a Gaussian filter to smooth the image to remove noise and details in the image, then use the finite difference of the first-order partial derivative to calculate the amplitude and direction of the gradient to detect edge points in the image, perform non-maximum suppression on the gradient amplitude to eliminate non-edge points, and finally use a double threshold algorithm to detect and connect edges to obtain continuous edge lines;

[0071] Step S43: Determine the area of ​​the air valve region: In order to better obtain the quantitative data of the air valve state, the picture of the air valve fully open and the picture of the air valve fully closed are respectively input into the algorithm to calculate the area of ​​the black area, which are recorded as A_open and A_closed respectively;

[0072] For the damper image in any state, calculate the black area A_current, and then use linear interpolation or other mathematical models to quantify the degree of opening and closing of the blade Valve_Opening_Degree;

[0073] The formula is shown in formula 3:

[0074]

[0075] Step S44: Calculate the average accuracy; in order to accurately quantify the opening degree of the air valve, analyze the individual air valve in each image to determine its opening degree in the single image, identify and exclude outliers through comprehensive use of statistical methods, and use robust statistical methods to calculate the average value of the remaining data, thereby obtaining an accurate and reliable opening degree of each air valve.

Claims

1. A method for detecting and identifying air valves based on deep learning, characterized in that: The method comprises the following steps: Step S1: Acquire the air valve image; Step S2: Use the yolox target detection model to detect the air valve image and obtain the type and position information of the air valve in the image; Step S3: The acquired position information of the air valve is passed into the trained binary classification model residual network ResNet50 to obtain the switch state of the air valve; Step S4: Perform image preprocessing on the air valve of the open type to obtain the edge feature information of the air valve blades, find the contour with the largest area among all external contours through contour detection, calculate the area of ​​the black area, and estimate the degree of opening and closing of the air valve based on the pre-defined minimum area when the air valve is closed and the maximum area when the air valve is open.

2. The air valve detection and identification method based on deep learning according to claim 1, characterized in that: In S1, a picture of the air valve video to be analyzed is taken at intervals of five frames as an air valve image.

3. The air valve detection and identification method based on deep learning according to claim 1, characterized in that: Before detection, it is necessary to obtain the weight of the air valve, that is, to perform annotation training on the air valve data; annotate the air valve category on the collected original image, integrate the original image and the annotated data into an air valve detection data set, and train the Yolox target detection model, which specifically includes the following steps: Step S21: Collecting initial data: Using a fixed industrial camera to collect images of air valves in different states, at different times and at different locations to form an air valve image data set; Step S22: label each sub-image in the air valve image data set using LabelImg software, record the location and category information of the air valve, and save it in txt text format; Step S23: randomly dividing the labeled air valve image dataset into a training set and a test set in a ratio of 4:1, wherein the training set includes 801 images and the test set includes 198 images; Step S24: Use the transfer learning strategy to train the Yolox target detection model.

4. The air valve detection and identification method based on deep learning according to claim 3 is characterized in that: In step S24, the entire training process includes 300 epochs in total. In the first 50 epochs, a frozen training phase is implemented. In this phase, only the neck network Neck and the head network Head of the model are fine-tuned, while the backbone network Backdone is kept unchanged. In the frozen phase, the batch size is set to 16; the next 250 epochs are the unfreezing training phase, in which the entire network structure participates in the training, and the batch size is set to 8.

5. The air valve detection and identification method based on deep learning according to claim 1, characterized in that: The step S3 specifically comprises the following steps: Step S31: using 500 open air valve state diagrams and closed air valve state diagrams as a data set for the binary classification model residual network ResNet50, wherein the data set is divided into a training set and a test set in a 4:1 ratio, and the learning rate is set to 0.00001 during training, the batch size is 32, and the training is performed for 100 rounds; Step S31: classify the air valve position information obtained by the yolox target detection model to obtain the switch state and air valve position information of the air valve.

6. The air valve detection and identification method based on deep learning according to claim 1, characterized in that: The image preprocessing in step S4 is as follows: converting the image into a grayscale image by grayscale conversion, retaining the contour and texture information of the image, reducing the image noise by applying a filter, and using Otsu's method to determine the optimal threshold based on the histogram statistical analysis of the image, thereby converting the image into a binary image. The binarization process will enhance the contrast between the leaf and the background, so that the leaf area becomes black and the background becomes white.

7. The air valve detection and identification method based on deep learning according to claim 6 is characterized in that: The formula of the Otsu method is Formula 1: If the current threshold is x, where ω0 is the proportion of foreground points, ω1 = 1-ω0 is the proportion of background points, μ0 = foreground grayscale mean, μ1 = background grayscale mean; u = ω0*μ0+ω1*μ1 global grayscale mean: y=μ0(μ0-u)*(μ0-u)+ω1(μ1-u)*(μ1-u) Equation 1 After simplification, we get formula 2: y=ω0*(1-ω0)*(μ0-μ1)*(μ0-μ1) Equation 2 The objective function is y, the larger y is, the better the threshold value of x is.

8. The air valve detection and identification method based on deep learning according to claim 7 is characterized in that: In step S4, edge feature information of the air valve blade is obtained by using Canny edge detection, and the contour with the largest area among all external contours is found by using contour detection.

9. The air valve detection and identification method based on deep learning according to claim 8, characterized in that: In the step S4: the picture of the air valve fully open and the picture of the air valve fully closed are input into the algorithm to calculate the area of ​​the black area, which are recorded as A_open and A_closed respectively; For the damper image in any state, calculate the black area A_current, and then use linear interpolation or other mathematical models to quantify the degree of opening and closing of the blade Valve_Opening_Degree; The formula is shown in formula 3:

10. The air valve detection and identification method based on deep learning according to claim 9, characterized in that: In step S4, a single air valve in each image is analyzed to determine its opening degree in the single image, and outliers are identified and excluded by comprehensive statistical methods, and the average value of the remaining data is calculated using a robust statistical method, thereby obtaining an accurate and reliable opening degree of each air valve.