Image and data internet-of-things combined equipment monitoring method

Abstract

The invention relates to the technical field of Internet of Things and intelligent monitoring, in particular to an image and data Internet of Things combined equipment monitoring method, which comprises the following steps of S1, deploying a data acquisition layer on target equipment, and monitoring the running state of the equipment in real time; s2, transmitting the data acquired by the acquisition layer to a cloud server; and S3, processing and analyzing the data in the cloud server, the implementation of the equipment monitoring method promotes digital transformation in the industrial field, and provides powerful support for enterprises to realize intelligent manufacturing and digital transformation. Through collection, analysis and utilization of equipment data, an enterprise can better understand the operation condition and the production process of the equipment, resource configuration is optimized, and production efficiency and quality are improved. Meanwhile, data support is provided for decision making of enterprises, and the core competitiveness of the enterprises is improved.

Description

Technical Field

[0001] The present invention relates to the field of Internet of Things technology and intelligent monitoring technology, and in particular to a device monitoring method combining image and data Internet of Things. Background Art

[0002] In the field of industrial automation, equipment monitoring has always been a crucial link. Traditional equipment monitoring mainly relies on physical sensors, which can monitor the operating status of equipment in real time. However, with the increasing complexity of industrial environments and the diversification of equipment types, equipment monitoring based solely on physical sensors has gradually exposed some limitations. The installation location and number of physical sensors are limited, and they cannot fully perceive the external operating conditions of the equipment and actual environmental changes. For example, for some large equipment or equipment under complex working conditions, physical sensors may not be able to accurately obtain the full picture information of the equipment, resulting in blind spots in monitoring. In complex industrial environments, physical sensors are easily affected by factors such as electromagnetic interference, vibration, and dust, resulting in certain errors and uncertainties in the collected data. This may affect the judgment of the operating status of the equipment, and then affect the normal operation and maintenance of the equipment. For outdoor industrial products, their working environment is more severe, the degree of association with physical IoT is not high, and the data interaction is imperfect. Traditional monitoring methods often cannot meet the real-time monitoring needs of outdoor equipment, and cannot timely understand the operating conditions and environmental changes of the equipment. In order to solve these problems and improve the accuracy and reliability of equipment monitoring, a new equipment monitoring method is urgently needed to combine image recognition technology with data IoT technology. Summary of the invention

[0003] In view of this, the present invention aims at the deficiencies of the prior art and proposes a device monitoring method combining image and data IoT, aiming to solve at least one of the problems raised by the above background technology.

[0004] The present invention provides a device monitoring method combining image and data IoT, comprising the following steps:

[0005] S1. Deploy the data collection layer on the target device to monitor the operating status of the device in real time;

[0006] S2, transmit the data collected by the collection layer to the cloud server;

[0007] S3. Process and analyze the data in the cloud server.

[0008] In some embodiments, the data collection layer is deployed on the target device, and the collection layer for real-time monitoring of the operating status of the device includes physical sensors and video / image collection devices.

[0009] In some embodiments, the physical sensor includes a temperature sensor, a pressure sensor, a vibration sensor, and a humidity sensor.

[0010] In some embodiments, the video / image acquisition device is a high-definition camera, which captures the external operating conditions of the device and actual environmental changes. The high-definition camera should have night vision, dustproof and waterproof functions.

[0011] In some embodiments, the data collected by the collection layer is transmitted to the cloud server via a wireless network.

[0012] In some embodiments, the processing and analysis of the data in the cloud server includes cleaning, denoising, and standardizing the collected physical sensor data and image data.

[0013] In some embodiments, the processing and analysis of data in the cloud server also includes using a deep learning algorithm to perform intelligent analysis on image data, using a convolutional neural network to extract features from the image, and identifying the status of the device, abnormal conditions, and environmental change information through a classifier or target detection algorithm.

[0014] In some embodiments, physical sensor data is integrated with image recognition results to form a comprehensive data model.

[0015] In some embodiments, ResNet is selected to train image data, and for device monitoring scenarios, a large amount of labeled image data is collected, including device images in normal states, device images in abnormal states, and device images under different environmental conditions, for model training and optimization;

[0016] Use transfer learning technology to utilize models pre-trained on large-scale data sets to accelerate the model training process, and use quantization technology to convert the model parameters from floating point numbers to integers to reduce the model's storage space and computational complexity;

[0017] Use model pruning technology to remove minor neurons and connections in the model to improve the model's reasoning speed.

[0018] In some embodiments, the temperature sensor data and the heating condition of the device in the image are fused using a weighted average method or a Kalman filter algorithm; and the device state recognition is fused using DS evidence theory or a fuzzy logic method.

[0019] Compared with the prior art, the beneficial effect of the present invention is that by combining image recognition technology and physical sensor data, the equipment can be monitored from multiple dimensions, reducing the errors that may be caused by a single data source. Image recognition technology can provide external image information of the equipment to help determine the operating status and abnormal conditions of the equipment; physical sensor data can provide internal parameter information of the equipment to reflect the actual operating status of the equipment. The combination of the two can more accurately evaluate the health status of the equipment and improve the reliability of equipment monitoring. Image recognition technology can monitor the external operating conditions of the equipment in real time and promptly discover abnormal performance of the equipment, such as foreign body intrusion, equipment damage, leakage, etc. By analyzing the image, the fault location can be quickly located, providing strong evidence for equipment fault diagnosis. At the same time, combined with physical sensor data and data analysis algorithms, the cause of the fault can be further analyzed, targeted maintenance suggestions can be provided, and the equipment fault repair time can be shortened. The analysis results based on the comprehensive data model can provide a scientific decision-making basis for the operation and maintenance of the equipment. For example, based on the operating data and historical records of the equipment, the optimal maintenance time and maintenance cycle of the equipment can be predicted, and the maintenance plan can be arranged in advance to reduce the failure rate and maintenance cost of the equipment. At the same time, through the evaluation of equipment performance, the operating parameters and process flow of the equipment can be optimized to improve the production efficiency and quality of the equipment. The implementation of this equipment monitoring method will promote the digital transformation of the industrial field and provide strong support for enterprises to achieve intelligent manufacturing and digital transformation. By collecting, analyzing and using equipment data, enterprises can better understand the operation of equipment and production processes, optimize resource allocation, and improve production efficiency and quality. At the same time, it also provides data support for enterprise decision-making and enhances the core competitiveness of enterprises.

[0020] The foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

[0021] Other features and aspects of the present disclosure will become more apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 Flowchart of a device monitoring method combining image and data IoT provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0025] In the description of the present application, it should be understood that the terms "center", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0026] The terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, unless otherwise specified, "plurality" means two or more.

[0027] In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0028] As mentioned in the background technology, in the field of industrial automation, equipment monitoring has always been a crucial link. Traditional equipment monitoring mainly relies on physical sensors, which can monitor the operating status of equipment in real time. However, with the increasing complexity of industrial environments and the diversification of equipment types, relying solely on physical sensors for equipment monitoring has gradually exposed some limitations. The installation location and number of physical sensors are limited, and it is impossible to fully perceive the external operating conditions of the equipment and actual environmental changes. For example, for some large equipment or equipment under complex working conditions, physical sensors may not be able to accurately obtain the overall information of the equipment, resulting in blind spots in monitoring. In a complex industrial environment, physical sensors are easily affected by factors such as electromagnetic interference, vibration, and dust, resulting in certain errors and uncertainties in the collected data. This may affect the judgment of the operating status of the equipment, and then affect the normal operation and maintenance of the equipment. For outdoor industrial products, their working environment is more severe, the degree of association with physical IoT is not high, and the data interaction is imperfect. Traditional monitoring methods are often difficult to meet the real-time monitoring needs of outdoor equipment, and it is impossible to understand the operating conditions and environmental changes of the equipment in a timely manner. In order to solve these problems and improve the accuracy and reliability of equipment monitoring, a new equipment monitoring method is urgently needed to combine image recognition technology with data IoT technology.

[0029] To improve the above problems, the present application proposes a device monitoring method combining image and data IoT. By combining image recognition technology and physical sensor data, the device can be monitored from multiple dimensions, reducing the errors that may be caused by a single data source. Image recognition technology can provide external image information of the device to help determine the operating status and abnormal conditions of the device; physical sensor data can provide internal parameter information of the device to reflect the actual operating status of the device. The combination of the two can more accurately assess the health status of the device and improve the reliability of equipment monitoring. Image recognition technology can monitor the external operating conditions of the device in real time and promptly detect abnormal performance of the device, such as foreign body intrusion, equipment damage, leakage, etc. By analyzing the image, the fault location can be quickly located, providing strong evidence for equipment fault diagnosis. At the same time, combined with physical sensor data and data analysis algorithms, the cause of the fault can be further analyzed, targeted maintenance suggestions can be provided, and the equipment fault repair time can be shortened. The analysis results based on the comprehensive data model can provide a scientific decision-making basis for the operation and maintenance of the equipment. For example, based on the operating data and historical records of the equipment, the optimal maintenance time and maintenance cycle of the equipment can be predicted, and the maintenance plan can be arranged in advance to reduce the failure rate and maintenance cost of the equipment. At the same time, by evaluating the performance of the equipment, the operating parameters and process flow of the equipment can be optimized, and the production efficiency and quality of the equipment can be improved. The implementation of this equipment monitoring method will promote the digital transformation of the industrial field and provide strong support for enterprises to achieve intelligent manufacturing and digital transformation. By collecting, analyzing and utilizing equipment data, enterprises can better understand the operation of the equipment and the production process, optimize resource allocation, and improve production efficiency and quality. At the same time, it also provides data support for the decision-making of enterprises and enhances the core competitiveness of enterprises.

[0030] See also Figure 1 As shown, according to an embodiment of the present application, a device monitoring method combining image and data IoT includes the following steps:

[0031] S1. Deploy the data collection layer on the target device to monitor the operating status of the device in real time;

[0032] S2, transmit the data collected by the collection layer to the cloud server;

[0033] S3. Process and analyze the data in the cloud server.

[0034] In some specific embodiments, the data collection layer is deployed on the target device, and the collection layer for real-time monitoring of the running status of the device includes physical sensors and video / image collection devices.

[0035] In some specific embodiments, the physical sensor includes a temperature sensor, a pressure sensor, a vibration sensor, and a humidity sensor.

[0036] In some specific embodiments, the video / image acquisition device is a high-definition camera, which captures the external operating conditions of the device and actual environmental changes. The high-definition camera should have night vision, dust and water resistance functions.

[0037] It should be understood that high-definition cameras can provide higher-resolution images, which means that more details can be captured when monitoring the external operation of the equipment. For example, abnormal conditions such as tiny cracks, corrosion, and foreign body intrusion of the equipment can be clearly recorded, which helps to detect potential problems in a timely manner. For image recognition algorithms, high-definition images provide richer feature information, which can improve the accuracy and reliability of recognition. For example, when identifying the model and logo of the equipment or judging the operating status of the equipment, high-definition images can allow the algorithm to classify and analyze more accurately. In industrial environments, many devices need to operate at night or in low light conditions. High-definition cameras with night vision function can still provide clear images in these situations, ensuring that equipment monitoring is not limited by time and achieving all-weather monitoring. For some outdoor equipment or important equipment, night is a high-incidence period for safety accidents. The night vision function can help the monitoring system better monitor the surrounding environment of the equipment at night, detect abnormal personnel or behavior in time, and improve the safety of the equipment. Industrial sites often have a lot of harsh environmental factors such as dust, dirt and water vapor. Dust-proof and waterproof high-definition cameras can effectively prevent these factors from eroding the camera lens and internal circuits, ensuring that the camera operates stably in harsh environments and extending the service life of the camera. Impurities such as dust and water vapor attached to the camera lens will cause blurred images and affect the monitoring effect. The dust and water-proof function can prevent the accumulation of these impurities, always maintain the clarity of the image, and ensure the accuracy of equipment monitoring.

[0038] In some specific embodiments, the data collected by the collection layer is transmitted to the cloud server via a wireless network.

[0039] In some specific embodiments, the processing and analysis of the data in the cloud server includes cleaning, denoising, and standardizing the collected physical sensor data and image data.

[0040] It should be understood that the data cleaning and denoising process can significantly improve the accuracy of data, remove noise and outliers, ensure the reliability of data, and provide a more accurate basis for subsequent analysis. Through standardization, data from different sources can be unified in format and enhance the compatibility of data. This helps to better integrate multi-source data in the data analysis process, improve data processing efficiency and the accuracy of analysis results. The data cleaning and denoising process can effectively improve the overall quality of data and reduce errors and redundant information. High-quality data is essential for training machine learning models and conducting advanced data analysis, and directly affects the credibility of analysis results. Standardized data is easier to store and retrieve, which improves the efficiency of data management. The structured data format makes database query and data call more efficient and reduces the time cost of data processing. Cleaned and standardized data is more consistent and suitable for a variety of analysis methods, such as statistical analysis, machine learning, and deep learning. Diversified analysis methods can explore the value of data from different angles and provide comprehensive support for decision-making.

[0041] In some specific embodiments, the processing and analysis of data in the cloud server also includes using a deep learning algorithm to perform intelligent analysis on image data, using a convolutional neural network to extract features from images, and identifying the status of the device, abnormal conditions, and environmental change information through a classifier or target detection algorithm.

[0042] It should be understood that the use of deep learning algorithms for intelligent analysis of image data can significantly improve the accuracy of equipment monitoring. By extracting image features through convolutional neural networks (CNN), automatic identification of equipment status, abnormal conditions and environmental changes can be achieved, thereby improving monitoring efficiency. Convolutional neural networks (CNN) have powerful feature extraction capabilities in image processing. By performing multi-level feature extraction on image data, subtle equipment anomalies and environmental changes can be captured, providing reliable data for subsequent classification and detection. Using classifiers or target detection algorithms to analyze images can accurately identify the status, abnormal conditions and environmental changes of equipment. This method improves the automation of the monitoring system, reduces the need for manual intervention, and improves the response speed. Deep learning algorithms support real-time monitoring and early warning functions. By continuously analyzing image data, the system can detect equipment failures or environmental anomalies in a timely manner and issue alarms to help operation and maintenance personnel take quick measures to avoid accidents. Deep learning algorithms are not only used for real-time monitoring, but can also analyze and optimize historical data. Through learning and model training of large amounts of image data, the accuracy and robustness of the algorithm are continuously improved, and the overall monitoring effect and equipment management level are improved.

[0043] In some specific embodiments, physical sensor data is integrated with image recognition results to form a comprehensive data model.

[0044] It should be understood that by integrating physical sensor data with image recognition results, the accuracy of equipment monitoring can be significantly improved. Physical sensors provide quantitative parameter information, such as temperature, pressure, etc., while image recognition provides information about the external state of the equipment and environmental changes. The combination of the two can more comprehensively reflect the operating status of the equipment and reduce false positives and false negatives. The comprehensive data model can better perform fault diagnosis. Physical sensor data can reveal abnormalities inside the equipment, while image recognition can detect external signs of faults, such as cracks, leaks, etc. This multi-dimensional information fusion enables the system to more accurately determine the type and severity of the fault and improve maintenance efficiency. The analysis results based on the comprehensive data model can provide a more scientific decision-making basis for equipment maintenance and care. For example, based on the operating parameters of the equipment and changes in the external environment, the maintenance needs of the equipment can be predicted, and the maintenance plan can be reasonably arranged to avoid excessive or insufficient maintenance.

[0045] In some specific embodiments, ResNet is selected to train image data, and for device monitoring scenarios, a large amount of labeled image data is collected, including device images in normal states, device images in abnormal states, and device images under different environmental conditions, for model training and optimization;

[0046] Use transfer learning technology to utilize models pre-trained on large-scale data sets to accelerate the model training process, and use quantization technology to convert the model parameters from floating point numbers to integers to reduce the model's storage space and computational complexity;

[0047] Use model pruning technology to remove minor neurons and connections in the model to improve the model's reasoning speed.

[0048] It should be understood that by collecting a large amount of annotated image data, including images of devices in normal states, abnormal states, and different environmental conditions, a rich and diverse training sample can be provided for the model. This helps ResNet learn and identify various device states and environmental changes more accurately, improving the accuracy and robustness of the model. Using transfer learning technology, a ResNet model pre-trained on a large-scale dataset can be used as a basis. This can reduce the time and resources required to train the model from scratch, quickly adapt to device monitoring scenarios, and accelerate the model training process. Quantization technology can convert the parameters of the model from floating point numbers to integers, thereby reducing the storage space and computational complexity of the model. This is very useful for deploying models on devices with limited resources, which can improve the operating efficiency and response speed of the model. Model pruning technology can remove minor neurons and connections in ResNet and reduce the complexity of the model. This can increase the inference speed of the model, enabling it to process image data faster and respond in a timely manner in real-time device monitoring scenarios. By training under multiple device states and environmental conditions, ResNet can learn a wider range of feature representations. This enables the model to have better generalization capabilities in different monitoring scenarios and adapt to various complex practical application scenarios. Accurate equipment monitoring can help detect equipment failures and abnormal conditions in a timely manner, take maintenance measures in advance, and avoid losses caused by equipment failures. At the same time, monitoring environmental changes around equipment can also improve workplace safety and protect the health and safety of personnel. The ResNet-based equipment monitoring system can realize automated image analysis and equipment status monitoring, reducing the need for manual intervention. This helps improve production efficiency, reduce operating costs, and promote the development of automation and intelligence in the industrial field.

[0049] In some specific embodiments, the temperature sensor data and the heating condition of the device in the image are fused by weighted averaging or Kalman filtering algorithm; and the device state recognition is fused by DS evidence theory or fuzzy logic method.

[0050] It should be understood that by fusing the temperature sensor data with the heating condition of the device in the image, the information of the two data sources can be integrated to reduce the errors and uncertainties that may exist in a single data source. For example, the temperature sensor may be affected by environmental factors, while the image data may be deviated due to problems such as light and angle. The fused data can more accurately reflect the actual heating condition of the device. The robustness of the system can be improved by fusing the temperature sensor data and the image data using the weighted average method or the Kalman filter algorithm. These algorithms can effectively handle the noise and outliers in the data, making the fused data more stable and reliable. Even in a complex industrial environment, the system can accurately monitor the heating condition of the device. Different data sources have different characteristics and advantages. The temperature sensor can provide accurate temperature values, while the image data can intuitively display the external state and heating parts of the device. By fusing these two types of data, the complementarity of multi-source information can be achieved, and the operating status of the device can be more comprehensively understood. For example, when the temperature sensor detects an increase in temperature, the specific location and cause of the heating can be further confirmed through the image data. For the state recognition of the device, the DS evidence theory or fuzzy logic method is used for fusion, which can combine multiple evidences and rules to improve the accuracy of state recognition. DS evidence theory can handle uncertain and incomplete information, and draw more reliable conclusions by combining information from multiple evidence sources. Fuzzy logic methods can handle fuzzy and uncertain situations, and make inferences and judgments based on fuzzy rules, which is suitable for complex equipment status recognition scenarios. The fused data can provide more powerful decision support for the operation, maintenance and management of equipment. Through the fusion analysis of temperature sensor data and image data, abnormal conditions and potential failures of equipment can be discovered in time, and preventive measures can be taken in advance to avoid accidents. At the same time, based on the status recognition results of fused data, more scientific and reasonable maintenance plans and strategies can be formulated to improve the reliability and service life of equipment.

[0051] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A device monitoring method combining image and data IoT, characterized in that: The following steps are involved: S1. Deploy the data collection layer on the target device to monitor the operating status of the device in real time; S2, transmit the data collected by the collection layer to the cloud server; S3. Process and analyze the data in the cloud server.

2. According to claim 1, a device monitoring method combining image and data IoT is characterized in that: The data collection layer is deployed on the target device, and the collection layer in the real-time monitoring of the running status of the device includes physical sensors and video / image collection devices.

3. The device monitoring method combining image and data IoT according to claim 2 is characterized in that: The physical sensors include temperature sensors, pressure sensors, vibration sensors, and humidity sensors.

4. The device monitoring method combining image and data IoT according to claim 3 is characterized in that: The video / image acquisition device is a high-definition camera, which captures the external operation status of the device and actual environmental changes. The high-definition camera should have night vision, dustproof and waterproof functions.

5. The device monitoring method combining image and data IoT according to claim 1 is characterized in that: The data collected by the collection layer is transmitted to the cloud server via wireless network.

6. The device monitoring method combining image and data IoT according to claim 1 is characterized in that: The processing and analysis of the data in the cloud server includes cleaning, denoising and standardizing the collected physical sensor data and image data.

7. The device monitoring method combining image and data IoT according to claim 6 is characterized in that: The processing and analysis of the data in the cloud server also includes using a deep learning algorithm to perform intelligent analysis on the image data, using a convolutional neural network to extract features from the image, and identifying the status of the device, abnormal conditions, and environmental change information through a classifier or target detection algorithm.

8. The device monitoring method combining image and data IoT according to claim 7 is characterized in that: Integrate physical sensor data with image recognition results to form a comprehensive data model.

9. The device monitoring method combining image and data IoT according to claim 7 is characterized in that: Select ResNet to train image data. For equipment monitoring scenarios, collect a large amount of labeled image data, including images of equipment in normal state, equipment in abnormal state, and equipment under different environmental conditions, for model training and optimization. Use transfer learning technology to utilize models pre-trained on large-scale data sets to accelerate the model training process, and use quantization technology to convert the model parameters from floating point numbers to integers to reduce the model's storage space and computational complexity; Use model pruning technology to remove minor neurons and connections in the model to improve the model's reasoning speed.

10. The device monitoring method combining image and data IoT according to claim 8 is characterized in that: For the temperature sensor data and the heating condition of the equipment in the image, the weighted average method or Kalman filter algorithm is used for fusion; for the state identification of the equipment, the DS evidence theory or fuzzy logic method is used for fusion.

Images