A non-invasive device monitoring method based on optical information

Abstract

The application relates to a non-invasive device monitoring method based on light information, and the method comprises the following steps: S1, acquiring light information data of a light source in an environment based on a light sensor or a camera, and obtaining a light signal after data preprocessing; S2, training a monitoring model based on a data set constructed based on the light signal, and obtaining a trained monitoring model; and S3, monitoring a light source signal in a power grid environment based on the trained monitoring model, and obtaining the working state of each monitoring object in the power grid environment. Compared with the prior art, the working state of an electrical device in the environment is monitored by monitoring the light source in the power grid and analyzing the light information of the light source, so that non-invasive monitoring is realized.

Description

Technical Field

[0001] This invention relates to the field of intelligent sensing, specifically a non-invasive device monitoring method based on optical information. Background Technology

[0002] The era of the Internet of Everything has begun. The ubiquitous power Internet of Things (IoT) fully utilizes modern information technologies such as mobile internet and artificial intelligence to achieve informatization and intelligentization in all aspects of the power system. A key factor is whether the sensing layer, the cornerstone of the ubiquitous power IoT, can be widely deployed to enterprises, industries, transportation, and even homes. For example, the vast industrial world consists of millions of machines and equipment, and thousands of complex machine networks. In this increasingly information-driven era, intelligent monitoring and management of machine and equipment operations are crucial to ensuring the safe and controllable operation of the entire machinery cluster and machine network, and are also the intrinsic driving force behind intelligent manufacturing.

[0003] In the most primitive industrial environments, monitoring electrical equipment often requires manual monitoring, which is difficult to locate, inefficient, and inaccurate, leading to delayed detection of equipment malfunctions, significant losses, and even safety accidents. To address these issues, automated equipment monitoring systems have been extensively researched in recent years, but systems that are low-cost, easy to operate, and widely applicable still require further research and development. Traditional solutions often include: 1) Distributed sensor monitoring: placing multiple sensors on different devices for individual monitoring; this method has the disadvantages of complex system installation and high cost; 2) Centralized sensor monitoring: placing sensors at the main power supply to monitor total energy consumption and determine if equipment is malfunctioning; this method requires additional equipment and has slow response times; 3) Intrusive sensor monitoring: placing sensors inside the circuit to monitor current changes; this method has the disadvantages of complex system installation and the need to modify the circuit structure. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the existing technology by providing a non-intrusive device monitoring method based on optical information, which monitors the working status of electrical equipment in the environment by monitoring the light sources in the power grid and analyzing the light information of the light sources.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] A non-invasive device monitoring method based on optical information, comprising the following steps:

[0007] S1. Obtain light information data of light sources in the environment based on light sensors or cameras, and obtain light signals after data preprocessing;

[0008] S2. Train the monitoring model based on the dataset constructed from the optical signal to obtain the trained monitoring model;

[0009] S3. Based on the trained monitoring model, monitor the light source signals in the power grid environment to obtain the working status of each monitored object in the power grid environment;

[0010] The specific steps of S1 include:

[0011] S11. Acquire light information data of light sources in the environment based on a light sensor or camera. When a light sensor is used, the obtained light information data is a light time-domain signal, and proceed to S12. When a camera is used, the obtained light information data is a light source image. Perform the following steps on the light source image to obtain the light time-domain signal:

[0012] S111, Obtain multiple light source images;

[0013] S112. Extract the fluorescent lamp area from each light source image and set a bounding box. Extract the RGB values ​​of the pixels within the bounding box in the image. Pad the blank areas with zeros. Stitch together multiple zero-pasted light source images to obtain the optical time domain signal.

[0014] S12. Use Fourier transform to convert the optical time domain signal into an optical frequency domain signal, and use the optical frequency domain signal as an optical signal.

[0015] Furthermore, the specific steps of S2 include:

[0016] S21. A supervised learning model is adopted to construct a labeled dataset based on optical signals;

[0017] S22. Extract feature data from samples in the dataset based on a multi-attribute classification network, and obtain the classification result based on the feature data and the multi-attribute classifier of the multi-attribute classification network.

[0018] S23. For the classification results, design a loss function to quantify the difference between the corresponding labels and classification results of each sample, optimize and iterate the multi-attribute classification network to obtain a well-trained monitoring model.

[0019] Furthermore, the specific steps of S21 include:

[0020] S211. When no monitored device is working, collect the first sample of the optical signal of S1 and set all the labels of the sample to zero, indicating that no monitored device is working.

[0021] S212. When a single monitoring device is working, collect the second sample of the optical signal from S1, change the monitoring device, and set the label of the second sample to indicate that the corresponding single monitoring device is working. Repeat S212 to obtain multiple sets of second samples.

[0022] S213. When multiple monitoring devices are working simultaneously, a third sample of the optical signal from S1 is collected. The label of the third sample indicates that multiple monitoring devices are working. The label of the third sample is set as the label indicating that the corresponding multiple monitoring devices are working. S212 is repeated to obtain multiple sets of third samples.

[0023] S214. Classify, package, and integrate the first sample, the second sample, and the third sample to generate a dataset.

[0024] Furthermore, a portion of the dataset is split off as a cross-validation set or a test set as needed.

[0025] Furthermore, the specific steps of S3 are as follows:

[0026] S31. Acquire light source signals in the power grid environment at pre-configured time intervals to obtain light information samples;

[0027] S32. Input the collected light information samples into the trained monitoring model, and obtain the working status of each monitoring object in the power grid environment by reading the output of the model.

[0028] Furthermore, the loss function is the multi-class cross-entropy loss function.

[0029] Furthermore, the stochastic gradient descent method is used to optimize and iterate the multi-attribute classification network.

[0030] Furthermore, the proposed multi-attribute classification network is based on an improved residual neural network, which includes a residual network ResNet_50, a region proposal network, a RoI pooling layer, a fully connected layer, and a Softmax multi-attribute classifier.

[0031] Furthermore, the specific steps of S22 include: extracting feature data from samples in the dataset based on the residual network ResNet_50 to obtain the corresponding feature maps; generating region proposals through the region proposal network; labeling the monitoring object features of the samples in the dataset; the RoI pooling layer extracting feature maps of the proposed regions based on the monitoring object features; inputting the feature maps of the proposed regions into the fully connected layer to extract feature data; and inputting the feature data into the multi-attribute classifier to obtain the classification results.

[0032] Furthermore, the optical sensor is an ADPD2212 sensor.

[0033] Compared with the prior art, the present invention has the following advantages:

[0034] (1) This invention does not require modification of the circuit structure or electrical equipment; it only requires the use of optical sensors to monitor the existing lighting equipment in the power grid. It is not only easy to monitor, but also does not change the structure of the existing machines or equipment, does not cause new problems or safety risks, and is simple and convenient to deploy and use.

[0035] (2) This invention can use a mobile phone camera to replace a light sensor to collect light frequency information from a light source. Light information is readily available and easy to collect; light sensors are inexpensive and easy to use; and the use of a smartphone monitoring system is simple to operate, widely applicable, and can also realize the data collection and online real-time analysis of light information.

[0036] (3) The present invention uses appropriate deep learning and artificial intelligence algorithms to train the model, making the analysis results more reliable. Attached Figure Description

[0037] Figure 1 This is a flowchart of the present invention;

[0038] Figure 2 This is a schematic diagram of the system working principle of the present invention;

[0039] Figure 3 This is a structural diagram of the model of the present invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0041] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0042] Example 1

[0043] This invention provides a non-invasive device monitoring method based on optical information, the flowchart of which is shown below. Figure 1 As shown. Figure 2 This is a schematic diagram illustrating the system's working principle. The method includes the following steps:

[0044] S1. Obtain light information data of light sources in the environment based on light sensors or cameras, and obtain light signals after data preprocessing.

[0045] S2. Train the monitoring model using a dataset constructed based on optical signals to obtain a trained monitoring model.

[0046] S3. Based on the trained monitoring model, monitor the light source signals in the power grid environment to obtain the working status of each monitored object in the power grid environment.

[0047] The specific steps of S1 include:

[0048] S11. Acquire light information data from ambient light sources using a light sensor or camera. Depending on the acquisition device, different types of data may be generated for processing. When using a light sensor, the obtained light information data is a time-domain optical signal S. t Execute S12; when using a camera, the obtained light information data is the light source image I. Perform the following steps on the light source image to obtain the light temporal signal S. t :

[0049] S111. Acquire multiple light source images I. This can be done using a rolling shutter camera. However, due to the limited resolution and insufficient spectral resolution of a single image, it is necessary to continuously capture multiple light source images I1, I2, ... I. n Multiple photos were then stitched together to improve resolution.

[0050] S112. Extract the fluorescent lamp region from each light source image I and set a bounding box. Extract the RGB values ​​of the pixels within the bounding box in the image. Pad the blank areas with zeros. Stitch together multiple zero-pasted light source images to obtain the optical temporal signal S. t ;

[0051] S12. Using Fourier transform to convert the optical time-domain signal S t Converted into optical frequency domain signal S f The optical frequency domain signal S f As an optical signal. Optical time-domain signal S t and optical frequency domain signal S f Both can be used for monitoring. In practical applications, frequency domain signals have more obvious characteristics, so their use is more recommended.

[0052] The optical sensor used in S11 can be the ADPD2212 sensor.

[0053] The goal of S2 is to train a model based on real-world application needs. The specific steps of S2 include:

[0054] S21. Using a supervised learning model, construct a labeled dataset D based on optical signals. train .

[0055] S22. Based on a multi-attribute classification network, feature data is extracted from samples S in the dataset. The classification result is obtained based on the feature data and the multi-attribute classifier of the multi-attribute classification network. The goal of S22 is to extract feature data F from the light signal samples S that helps the classifier make decisions, and to train the multi-classifier based on the feature data to output the classification result O.

[0056] S23. For the classification result O, design a loss function to quantify the difference between the corresponding label and the classification result of each sample, and optimize and iterate the multi-attribute classification network to obtain a trained monitoring model. During the optimization and iteration process, by adjusting the method of constructing sample labels in step S21, the trainer can achieve different functions such as device recognition and anomaly detection.

[0057] The specific steps in S21 include:

[0058] S211. The first sample S of the optical signal collected when no monitoring device is operating. bg The sample labels were set to zero to indicate that no monitored object was working. The sample contained white noise and light signal characteristics of a fluorescent lamp source when it was working normally.

[0059] S212. When a single monitoring device is operating, the second sample S of the optical signal collected from S1 is... sin The monitoring target device is changed, and the label of the second sample is set to indicate that the corresponding single monitoring target device is working. This process is repeated with the changed operating device to obtain multiple sets of second samples. The labels of the second samples can be set to be more granular, such as indicating normal device operation or abnormal device operation, depending on actual needs.

[0060] S213. When multiple monitoring devices are working simultaneously, the third sample S of the optical signal collected from S1 is... mul The label of the third sample indicates that multiple monitoring devices are working. The label of the third sample is set to indicate that the corresponding multiple monitoring devices are working. The running devices are changed and S212 is repeated to obtain multiple sets of third samples.

[0061] S214. Classify, package, and integrate the first, second, and third samples to generate dataset D. train The obtained dataset D train A portion can be segmented as a cross-validation set or test set according to requirements.

[0062] The S22 multi-attribute classification network is an improvement on the Residual Neural Network (ResNet). The multi-attribute classification network includes the ResNet_50 residual network, the Region Proposal Network, the RoI pooling layer, the fully connected layer, and the Softmax multi-attribute classifier. Figure 3 This is a diagram of the multi-attribute classification network structure of the present invention.

[0063] The specific steps of S22 include: extracting feature data from samples in the dataset based on the residual network ResNet_50 to obtain the corresponding feature maps; generating region proposals through the region proposal network; labeling the monitoring object features of the samples in the dataset; the RoI pooling layer extracting feature maps of the proposed regions based on the monitoring object features; inputting the feature maps of the proposed regions into the fully connected layer to extract feature data; and inputting the feature data into the Softmax multi-attribute classifier to obtain the classification result O.

[0064] The loss function of S23 is the multi-class cross-entropy loss function. S23 employs stochastic gradient descent to iteratively optimize the multi-attribute classification network.

[0065] S3 utilizes the monitoring model trained in step S2 to continuously monitor light source signals in the power grid environment and outputs the operating status of each monitored object in the power grid environment. The specific steps of S3 are as follows:

[0066] S31. Acquire light source signals in the power grid environment at pre-configured time intervals to obtain optical information samples S. e ;

[0067] S32. Input the collected optical information samples into the trained monitoring model, and obtain the working status of each monitored object in the power grid environment by reading the model's output. After execution, repeating steps S31 and S32 can achieve the effect of real-time monitoring and timely early warning.

[0068] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A non-invasive device monitoring method based on optical information, characterized in that, The method includes the following steps: S1. Obtain light information data of light sources in the environment based on light sensors or cameras, and obtain light signals after data preprocessing; S2. Train the monitoring model based on the dataset constructed from the optical signal to obtain the trained monitoring model; S3. Based on the trained monitoring model, monitor the light source signals in the power grid environment to obtain the working status of each monitored object in the power grid environment; The specific steps of S1 include: S11. Acquire light information data of light sources in the environment based on a light sensor or camera. When a light sensor is used, the obtained light information data is a light time-domain signal, and proceed to S12. When a camera is used, the obtained light information data is a light source image. Perform the following steps on the light source image to obtain the light time-domain signal: S111, Obtain multiple light source images; S112. Extract the fluorescent lamp area from each light source image and set a bounding box. Extract the RGB values ​​of the pixels within the bounding box in the image. Pad the blank areas with zeros. Stitch together multiple zero-padding light source images to obtain the optical time domain signal. S12. Use Fourier transform to convert the optical time domain signal into an optical frequency domain signal, and use the optical frequency domain signal as an optical signal.

2. The non-invasive device monitoring method based on optical information according to claim 1, characterized in that, The specific steps of S2 include: S21. A supervised learning model is adopted to construct a labeled dataset based on optical signals; S22. Extract feature data from samples in the dataset based on a multi-attribute classification network, and obtain the classification result based on the feature data and the multi-attribute classifier of the multi-attribute classification network. S23. For the classification results, design a loss function to quantify the difference between the corresponding labels and classification results of each sample, optimize and iterate the multi-attribute classification network, and obtain a well-trained monitoring model.

3. The non-invasive device monitoring method based on optical information according to claim 2, characterized in that, The specific steps in S21 include: S211. When no monitored device is working, collect the first sample of the optical signal of S1 and set all the labels of the sample to zero, indicating that no monitored device is working. S212. When a single monitoring device is working, collect the second sample of the optical signal from S1, change the monitoring device, and set the label of the second sample to indicate that the corresponding single monitoring device is working. Repeat S212 to obtain multiple sets of second samples. S213. When multiple monitoring devices are working simultaneously, a third sample of the optical signal from S1 is collected. The label of the third sample indicates that multiple monitoring devices are working. The label of the third sample is set as the label indicating that the corresponding multiple monitoring devices are working. S212 is repeated to obtain multiple sets of third samples. S214. Classify, package, and integrate the first sample, the second sample, and the third sample to generate a dataset.

4. The non-invasive device monitoring method based on optical information according to claim 3, characterized in that, The dataset is divided into a portion as a cross-validation set or a test set according to requirements.

5. The non-invasive device monitoring method based on optical information according to claim 1, characterized in that, The specific steps for S3 are as follows: S31. Acquire light source signals in the power grid environment at pre-configured time intervals to obtain light information samples; S32. Input the collected light information samples into the trained monitoring model, and obtain the working status of each monitoring object in the power grid environment by reading the output of the model.

6. The non-invasive device monitoring method based on optical information according to claim 2, characterized in that, The loss function is the multi-class cross-entropy loss function.

7. The non-invasive device monitoring method based on optical information according to claim 2, characterized in that, The stochastic gradient descent method is used to optimize and iterate the multi-attribute classification network.

8. The non-invasive device monitoring method based on optical information according to claim 2, characterized in that, The proposed multi-attribute classification network is based on an improved residual neural network. The multi-attribute classification network includes a residual network ResNet_50, a region proposal network, a RoI pooling layer, a fully connected layer, and a Softmax multi-attribute classifier.

9. A non-invasive device monitoring method based on optical information according to claim 8, characterized in that, The specific steps of S22 include: extracting feature data from samples in the dataset based on the residual network ResNet_50 to obtain the corresponding feature maps; generating region proposals through the region proposal network; labeling the monitoring object features of the samples in the dataset; the RoI pooling layer extracting feature maps of the proposed regions based on the monitoring object features; inputting the feature maps of the proposed regions into the fully connected layer to extract feature data; and inputting the feature data into the multi-attribute classifier to obtain the classification results.

10. A non-invasive device monitoring method based on optical information according to claim 1, characterized in that, The optical sensor is an ADPD2212 sensor.

Images