Equipment remote intelligent monitoring management system and method

Abstract

The invention discloses an equipment remote intelligent monitoring management system and method, and the system employs a camera to collect a state image of a charging pile, and transmits the image to a background server through a wireless transmission module. In the background server, an image processing and analysis algorithm based on machine vision is introduced to analyze the state image so as to judge whether the charging pile is damaged or not based on the on-site picture, such as component falling, abnormal display of a power indicator lamp and the like, so that remote intelligent monitoring management of the charging pile can be realized through the mode, and the charging pile is convenient to use. And the monitoring management efficiency and the service quality of the charging pile are improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent management, and more specifically, to a remote intelligent monitoring and management system and method for equipment. Background Art

[0002] With the popularity of electric vehicles, the number and frequency of use of charging piles are also increasing, and the safety and stability of charging piles have become an important issue. However, charging piles may experience abnormal conditions and damage during operation, such as component detachment and abnormally displayed power indicator lights, which may cause damage to the charging piles or cause safety accidents such as fires. In order to ensure the normal operation and safety of charging piles, effective monitoring and management of charging piles is crucial.

[0003] Traditional charging pile monitoring and management systems usually require manual inspections of the status of charging piles, which is time-consuming, labor-intensive and inefficient. At the same time, manual inspections are prone to missing or delaying the discovery of charging pile failures or damage, resulting in longer maintenance response times. In addition, traditional manual inspections limit the scope and flexibility of monitoring, reducing the effectiveness and reliability of charging pile monitoring and management.

[0004] Therefore, a remote intelligent monitoring and management system for equipment is desired. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a remote intelligent monitoring and management system and method for equipment, which uses a camera to collect the status image of the charging pile and sends the image to the background server through a wireless transmission module. In the background server, an image processing and analysis algorithm based on machine vision is introduced to analyze the status image, so as to judge whether the charging pile is damaged based on the on-site photos, for example, parts fall off, abnormal power indicator light, etc. In this way, remote intelligent monitoring and management of the charging pile can be realized, and the monitoring and management efficiency and service quality of the charging pile can be improved.

[0006] According to one aspect of the present application, a remote intelligent monitoring and management system for equipment is provided, which includes:

[0007] A charging pile status image acquisition module is used to acquire status images of monitored charging piles through a camera;

[0008] A state image transmission module, used to transmit the state image to a background server via a wireless transmission module;

[0009] A state image semantic segmentation module, used to perform foreground and background segmentation on the state image on the backend server to obtain a charging pile foreground object area image;

[0010] A charging pile foreground image feature analysis module, used for performing feature analysis on the charging pile foreground object area image to obtain charging pile foreground features;

[0011] The charging pile damage detection module is used to determine whether the monitored charging pile is damaged based on the foreground characteristics of the charging pile.

[0012] According to another aspect of the present application, a method for remote intelligent monitoring and management of equipment is provided, which includes:

[0013] Collect the status image of the monitored charging pile through the camera;

[0014] Transmitting the status image to a backend server via a wireless transmission module;

[0015] On the backend server, segmenting the state image into foreground and background to obtain a charging pile foreground object area image;

[0016] Performing feature analysis on the charging pile foreground object area image to obtain charging pile foreground features;

[0017] Based on the foreground features of the charging pile, determine whether the monitored charging pile is damaged.

[0018] Compared with the prior art, the present application provides a remote intelligent monitoring management system and method for equipment, which uses a camera to collect the status image of the charging pile and sends the image to the background server through a wireless transmission module. In the background server, an image processing and analysis algorithm based on machine vision is introduced to analyze the status image, so as to judge whether the charging pile is damaged based on the on-site photos, for example, parts fall off, abnormal power indicator light, etc. In this way, remote intelligent monitoring and management of the charging pile can be realized, and the monitoring and management efficiency and service quality of the charging pile can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0020] Figure 1 A block diagram of a remote intelligent monitoring and management system for equipment according to an embodiment of the present application;

[0021] Figure 2 A system architecture diagram of a remote intelligent monitoring and management system for equipment according to an embodiment of the present application;

[0022] Figure 3 A block diagram of the training phase of the remote intelligent monitoring and management system for equipment according to an embodiment of the present application;

[0023] Figure 4 It is a block diagram of a charging pile foreground image feature analysis module in a remote intelligent monitoring and management system for equipment according to an embodiment of the present application;

[0024] Figure 5 It is a block diagram of a charging pile prospect feature enhancement unit in a remote intelligent monitoring and management system for equipment according to an embodiment of the present application;

[0025] Figure 6 Flow chart of a method for remote intelligent monitoring and management of equipment according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0027] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0028] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0029] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0030] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0031] Traditional charging pile monitoring and management systems usually require manual inspection of the status of charging piles, which is time-consuming, labor-intensive and inefficient. At the same time, manual inspections are prone to miss or delay the discovery of charging pile failures or damage, resulting in extended maintenance response time. In addition, traditional manual inspection methods limit the scope and flexibility of monitoring, reducing the effectiveness and reliability of charging pile monitoring and management. Therefore, a remote intelligent monitoring and management system for equipment is desired.

[0032] In the technical solution of the present application, a remote intelligent monitoring and management system for equipment is proposed. Figure 1 It is a block diagram of a remote intelligent monitoring and management system for equipment according to an embodiment of the present application. Figure 2 : is a system architecture diagram of a remote intelligent monitoring and management system for equipment according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to the embodiment of the present application, the remote intelligent monitoring and management system 300 of the equipment includes: a charging pile state image acquisition module 310, which is used to acquire the state image of the monitored charging pile through a camera; a state image transmission module 320, which is used to transmit the state image to the background server through a wireless transmission module; a state image semantic segmentation module 330, which is used to perform foreground and background segmentation on the state image on the background server to obtain a charging pile foreground object area image; a charging pile foreground image feature analysis module 340, which is used to perform feature analysis on the charging pile foreground object area image to obtain a charging pile foreground feature; a charging pile damage detection module 350, which is used to determine whether the monitored charging pile is damaged based on the charging pile foreground feature.

[0033] In particular, the charging pile status image acquisition module 310 is used to acquire the status image of the monitored charging pile through a camera. It should be understood that the status image of the charging pile can reflect the status, damage, etc. of the charging pile. Therefore, in a specific example, the status image of the monitored charging pile is obtained by deploying cameras around the charging pile. It is worth mentioning that during the selection and installation of the camera, a camera with a higher resolution should be selected, and the camera's all-round angle adjustability and stability should be ensured during installation, so as to obtain a clearer and more complete status image of the monitored charging pile, which is convenient for subsequent image analysis and classification tasks.

[0034] In particular, the state image transmission module 320 is used to transmit the state image to the background server through the wireless transmission module. That is, in the technical solution of the present application, the state image is transmitted to the background server through the wireless transmission module, so that the characteristics of the state image of the monitored charging pile can be captured and analyzed in the background server, and the remote monitoring and management of the charging pile can be realized, which is conducive to timely detection of abnormal conditions and damage of the charging pile, and improves the efficiency of monitoring and management of the charging pile.

[0035] In particular, the state image semantic segmentation module 330 is used to perform foreground and background segmentation on the state image in the background server to obtain a foreground object area image of the charging pile. It should be understood that the background around the charging pile may contain other objects or interference elements, which may interfere with the accuracy of fault detection. Therefore, when the background server performs feature capture and charging pile anomaly detection on the state image, in order to distinguish the charging pile from the surrounding environment to eliminate the interference of the background, so that the subsequent processing is more focused on the charging pile itself, in the technical solution of the present application, the state image is further segmented into foreground and background to obtain a charging pile foreground object area image. By separating the charging pile from the background, the foreground object area image of the charging pile can be extracted, so that the subsequent feature analysis focuses more on the feature information of the charging pile and ignores the background interference features that are not related to the abnormal detection of the charging pile. In other words, by separating the charging pile from the background, the feature information of the charging pile can be more accurately extracted. These feature information may include the shape, color, texture, etc. of the charging pile, which is helpful for subsequent fault detection and classification.

[0036] In particular, the charging pile foreground image feature analysis module 340 is used to perform feature analysis on the charging pile foreground object area image to obtain the charging pile foreground feature. In particular, in a specific example of the present application, Figure 4 As shown, the charging pile foreground image feature analysis module 340 includes: a charging pile foreground image feature extraction unit 341, which is used to extract features of the charging pile foreground object area image through an image feature extractor based on a deep neural network model to obtain a charging pile foreground feature map; a charging pile foreground feature enhancement unit 342, which is used to perform spatial feature enhancement and full-perception encoding on the charging pile foreground feature map to obtain the charging pile foreground feature.

[0037] Specifically, the charging pile foreground image feature extraction unit 341 is used to extract features from the charging pile foreground object area image through an image feature extractor based on a deep neural network model to obtain a charging pile foreground feature map. That is, in the technical solution of the present application, an image feature extractor based on a convolutional neural network model with excellent performance in implicit feature extraction of images is used to perform feature mining on the charging pile foreground object area image to extract implicit feature distribution information about the charging pile in the charging pile foreground object area image, such as shape, color, texture and other features, so as to more accurately detect abnormal situations such as the detachment of charging pile components and abnormally displayed power indicator lights, thereby obtaining a charging pile foreground feature map. More specifically, each layer of the image feature extractor based on the convolutional neural network model performs the following operations on the input data in the forward pass of the layer: convolution processing on the input data to obtain a convolutional feature map; pooling based on the local feature matrix on the convolutional feature map to obtain a pooled feature map; and nonlinear activation on the pooled feature map to obtain an activated feature map; wherein the output of the last layer of the image feature extractor based on the convolutional neural network model is the charging pile foreground feature map, and the input of the first layer of the image feature extractor based on the convolutional neural network model is the charging pile foreground object area image.

[0038] It is worth noting that Convolutional Neural Network (CNN) is a deep learning model that is mainly used to process data with grid structures, such as images and videos. CNN is widely used in the field of computer vision, especially for tasks such as image classification, object detection, and image segmentation. The core idea of ​​CNN is to build a deep network through convolutional layers, pooling layers, and fully connected layers to automatically learn feature representations in images. Below is the general structure and main components of CNN: Convolutional layer: The convolutional layer is the core component of CNN and is used to extract local features of an image. The convolutional layer generates a feature map by performing a convolution operation on the input image by defining a set of learnable filters (also called convolution kernels or convolution weights). Each filter can capture different features, such as edges, textures, or shapes; Activation function: An activation function, such as ReLU, is usually added after the convolutional layer to introduce nonlinear characteristics. The activation function increases the expressive power of the network by performing an element-by-element nonlinear transformation on each element of the feature map; Pooling layer: The pooling layer is used to reduce the size of the feature map and retain important features. The commonly used pooling operation is max pooling, which selects the maximum value in each pooling window as the pooling result. Pooling can reduce the amount of calculation and enhance the robustness of the model to translation and scale changes; Fully connected layer: After extracting features through multiple convolutional layers and pooling layers, a fully connected layer is usually added. The fully connected layer flattens the feature map into a one-dimensional vector and multiplies it with the weight matrix for tasks such as classification or regression; Dropout layer: In order to prevent overfitting, the Dropout layer is often used in CNN. The Dropout layer randomly discards the output of some neurons with a certain probability, thereby reducing the dependency between neurons and improving the generalization ability of the model; Softmax layer: In classification tasks, a Softmax layer is usually added to the output layer to convert the output of the network into a probability distribution. The Softmax function can map the original output of the network to the probability of each category so that the sum of the output values ​​is 1. The training process of CNN usually uses the backpropagation algorithm and the gradient descent optimization method to update the weight parameters of the network by minimizing the loss function. During the training process, CNN gradually learns better feature representation by continuously adjusting the weights of the convolution kernel.

[0039] Specifically, the charging pile foreground feature enhancement unit 342 is used to perform spatial feature enhancement and full perceptual encoding on the charging pile foreground feature map to obtain the charging pile foreground feature. In particular, in a specific example of the present application, Figure 5As shown, the charging pile foreground feature enhancement unit 342 includes: a charging pile foreground feature spatial manifestation subunit 3421, which is used to pass the charging pile foreground feature map through a spatial attention layer to obtain a spatially manifested charging pile foreground feature map; a charging pile feature global perception subunit 3422, which is used to pass the spatially manifested charging pile foreground feature map through a global sensor based on a non-local neural network model to obtain a globally perceived charging pile foreground feature map as the charging pile foreground feature.

[0040] More specifically, the charging pile foreground feature spatial manifestation subunit 3421 is used to pass the charging pile foreground feature map through the spatial attention layer to obtain a spatially manifested charging pile foreground feature map. It should be understood that in order to further highlight and strengthen the important features of the charging pile, such as the position area features of each component of the charging pile and the power indicator light display area feature information, so as to improve the accuracy and reliability of fault detection, in the technical solution of the present application, the charging pile foreground feature map is further passed through the spatial attention layer to obtain a spatially manifested charging pile foreground feature map. It should be understood that the spatial attention layer can focus more attention on the key areas of the charging pile by learning the spatial distribution pattern of the charging pile. In this way, the feature representation of the charging pile can be enhanced, so that the model pays more attention to the details and features of the charging pile, and improves the accuracy of fault detection. In a specific example, the charging pile foreground feature map is passed through a spatial attention layer to obtain a spatially explicit charging pile foreground feature map, including: passing the charging pile foreground feature map through the spatial attention module to obtain a weight score map; inputting the weight score map into a Softmax activation function to obtain a spatial attention weight feature map; and calculating the position point multiplication between the spatial attention weight feature map and the charging pile foreground feature map to obtain the spatially explicit charging pile foreground feature map.

[0041] It is worth noting that the spatial attention layer is a commonly used attention mechanism in convolutional neural networks to enhance the model's attention to important spatial locations in the input data. The spatial attention layer can help the model automatically learn and focus on the most relevant areas when processing images or other data with spatial structure. The spatial attention layer can be embedded in different layers of the convolutional neural network, such as after the convolution layer, after the pooling layer, or before the fully connected layer. By introducing the spatial attention mechanism, the model can adaptively focus on important areas in the input data, improving the model's expressiveness and performance.

[0042] More specifically, the charging pile feature global perception subunit 3422 is used to pass the spatially manifested charging pile foreground feature map through a global perceptron based on a non-local neural network model to obtain a globally perceived charging pile foreground feature map as the charging pile foreground feature. Considering that convolution is a typical local operation, it can only extract local features of the image, but cannot focus on the global, which will affect the detection accuracy. Therefore, in order to obtain a more global charging pile foreground feature map to further improve the accuracy and reliability of fault detection, in the technical solution of the present application, the spatially manifested charging pile foreground feature map is further passed through a global perceptron based on a non-local neural network model to obtain a globally perceived charging pile foreground feature map. In the charging pile monitoring system, the global perceptron is a model for capturing global contextual information. It can obtain a wider range of contextual information by analyzing the features of the entire image, so as to better understand and represent the feature distribution information of the charging pile, and more accurately judge the status and damage of the charging pile.

[0043] It is worth noting that the non-local neural network is a neural network structure used to model long-range dependencies. Its concept is to capture the global correlation information in the input data by introducing non-local blocks to improve the expressiveness and performance of the model. In traditional convolutional neural networks (CNNs), the features of each position are only related to the features of its neighboring positions. However, in some tasks, global dependencies may be crucial for accurate modeling and prediction. Non-local neural networks solve this problem by introducing non-local blocks. The core idea of ​​non-local blocks is to obtain global correlation information by calculating the similarity between different positions in the input feature map. Specifically, for each position, the non-local block calculates the similarity between it and all other positions and applies these similarities to the update and integration of features. It has a wide range of applications in tasks such as video analysis, target tracking, natural language processing, and image processing.

[0044] It is worth mentioning that in other specific examples of the present application, the charging pile foreground feature map can also be spatially enhanced and fully perceptually encoded in other ways to obtain the charging pile foreground feature, for example: the charging pile foreground feature map obtained by the foreground and background segmentation step; spatial feature enhancement aims to enhance the spatial information in the charging pile foreground feature map to better capture the shape and structure of the charging pile. This can be achieved by the following steps: edge detection: the charging pile foreground feature map is processed using an edge detection algorithm (such as Canny edge detection) to detect edge information; edge enhancement: the detected edges are enhanced, and filters can be used to enhance the strength and clarity of the edges; morphological operations: morphological operations (such as dilation and corrosion) are applied to further improve the connectivity and shape of the edges; feature enhancement: the enhanced edges are combined with the original charging pile foreground feature map to enhance the edge and detail information in the feature map. Full perceptual encoding: Full perceptual encoding aims to encode the charging pile foreground feature map into a more compact and high-dimensional feature representation to capture the global and local features of the charging pile. This can be achieved by the following steps: feature extraction: meaningful features are extracted from the charging pile foreground feature map using a convolutional neural network or other feature extraction model. These features can be the output of the convolutional layer or the activation value of the fully connected layer; Feature Dimensionality Reduction: Reduce the dimension of the extracted features to reduce the dimension and complexity of the features. Common dimensionality reduction methods include principal component analysis and autoencoders; Feature Fusion: Fuse the reduced features with other related features to obtain more comprehensive charging pile prospect features. This can include spatial information, color features, texture features, etc.; Feature Encoding: Encode the fused features, and use an encoder-decoder structure or other encoding methods to map the features to a more compact representation; Feature Normalization: Normalize the encoded features to ensure that the features have consistent importance at different scales and ranges.

[0045] It is worth mentioning that in other specific examples of the present application, the charging pile foreground object area image can also be feature analyzed in other ways to obtain the charging pile foreground features, for example: obtaining a charging pile foreground object area image dataset; preprocessing the image, such as resizing, cropping or standardizing, to ensure that the input image has a consistent feature representation; using an appropriate feature extraction method to extract useful features from the image. The following are some commonly used feature extraction methods: color-based: extracting features such as color histogram, color moment or color gradient in the image; texture-based: extracting texture features in the image, such as grayscale co-occurrence matrix, local binary pattern (LBP) or Gabor filter response; shape-based: extracting shape features of the charging pile, such as edge detection, contour descriptor or shape context; deep learning-based: using a convolutional neural network (CNN) or a pre-trained model (such as ResNet, VGG, etc.) to extract image features; converting the extracted features into a vector representation suitable for machine learning algorithms. Dimensionality reduction methods such as principal component analysis (PCA) or linear discriminant analysis (LDA) can be used to reduce feature dimensions and retain the most representative information; the extracted features can be analyzed using statistical analysis methods, visualization tools, or machine learning algorithms. For example, the correlation between features can be calculated, the distribution of features can be observed, or similar features can be grouped using clustering algorithms; the extracted and selected features can be applied to tasks related to the foreground object of the charging pile, such as charging pile recognition, target detection, or target tracking.

[0046] In particular, the charging pile damage detection module 350 is used to determine whether the monitored charging pile is damaged based on the foreground features of the charging pile. In particular, in the technical solution of the present application, the global perception charging pile foreground feature map is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether the monitored charging pile is damaged. That is to say, the foreground feature distribution information about the charging pile after spatial feature enhancement and feature global perception processing is used for classification processing, so as to evaluate whether the charging pile is damaged. In this way, it is possible to judge whether the charging pile is damaged based on on-site photos, for example, parts falling off, abnormally displayed power indicator lights, etc., to improve the monitoring management efficiency and service quality of the charging pile. Specifically, the global perception charging pile foreground feature map is expanded into a global perception charging pile foreground feature vector based on a row vector or a column vector; the classification feature vector is fully connected encoded using multiple fully connected layers of the classifier to obtain an encoded classification feature vector; and the encoded classification feature vector is passed through the Softmax classification function of the classifier to obtain the classification result.

[0047] A classifier is a machine learning model or algorithm that is used to classify input data into different categories or labels. Classifiers are part of supervised learning and perform classification tasks by learning the mapping relationship from input data to output categories.

[0048] A fully connected layer is a common type of layer in neural networks. In a fully connected layer, each neuron is connected to all neurons in the previous layer, and each connection has a weight. This means that each neuron in the fully connected layer receives inputs from all neurons in the previous layer, and performs a weighted sum of these inputs through the weights, and then passes the result to the next layer.

[0049] The Softmax classification function is a commonly used activation function for multi-classification problems. It converts each element of the input vector into a probability value between 0 and 1, and the sum of these probability values ​​is equal to 1. The Softmax function is often used in the output layer of a neural network, especially for multi-classification problems, because it can map the network output to the probability distribution of each category. During the training process, the output of the Softmax function can be used to calculate the loss function and update the network parameters through the back propagation algorithm. It is worth noting that the output of the Softmax function does not change the relative size relationship between the elements, but only normalizes them. Therefore, the Softmax function does not change the characteristics of the input vector, but only converts it into a probability distribution form.

[0050] It should be understood that before using the above-mentioned neural network model for inference, it is necessary to train the image feature extractor based on the convolutional neural network model, the spatial attention layer, the global perceptron based on the non-local neural network model, and the classifier. That is to say, according to the device remote intelligent monitoring and management system 300 of the present application, it also includes a training stage 400 for training the image feature extractor based on the convolutional neural network model, the spatial attention layer, the global perceptron based on the non-local neural network model, and the classifier.

[0051] Figure 3 FIG. 1 is a block diagram of the training phase of the remote intelligent monitoring and management system for equipment according to an embodiment of the present application. Figure 3As shown, according to the embodiment of the present application, the remote intelligent monitoring and management system 300 of the equipment includes: a training stage 400, including: a training data acquisition unit 410, used to obtain training data, the training data including a training state image of the monitored charging pile, and a real value of whether the monitored charging pile is damaged; a training data transmission unit 420, used to transmit the training state image to the background server through a wireless transmission module; a training state image foreground and background segmentation unit 430, used to perform foreground and background segmentation on the training state image on the background server to obtain a training charging pile foreground object area image; a training state image foreground feature extraction unit 440, used to extract the training charging pile foreground object area image through the image feature extractor based on the convolutional neural network model to obtain a training charging pile foreground feature extraction unit; feature map; a training image spatial feature manifestation unit 450, used to pass the training charging pile foreground feature map through the spatial attention layer to obtain a training spatially manifested charging pile foreground feature map; a training image feature global perception unit 460, used to pass the training spatially manifested charging pile foreground feature map through the global perceptron based on the non-local neural network model to obtain a training global-perception charging pile foreground feature map; a classification loss unit 470, used to pass the training global-perception charging pile foreground feature map through the classifier to obtain a classification loss function value; a model training unit 480, used to train the image feature extractor based on the convolutional neural network model, the spatial attention layer, the global perceptron based on the non-local neural network model and the classifier based on the classification loss function value and through the direction propagation of gradient descent.

[0052] Among them, the classification loss unit is used to: use the classifier to process the training global perception charging pile foreground feature map to obtain a training classification result; and calculate the cross entropy loss function value between the training classification result and the true value of whether the monitored charging pile is damaged as the classification loss function value.

[0053] In particular, in the technical solution of the present application, by performing foreground and background segmentation on the training state image, the interference of the background part on the charging pile detection can be removed, that is, the adverse effects of other objects in the image source domain on the charging pile detection can be reduced by image segmentation. When the foreground object area image of the training charging pile is passed through an image feature extractor based on a convolutional neural network model, the image local neighborhood semantic association features based on the convolution kernel scale of the foreground object area image of the training charging pile can be extracted, and the feature expression of certain spatial areas in the foreground feature map of the training charging pile can be strengthened through the spatial attention layer. Furthermore, when the foreground feature map of the training spatially manifested charging pile is passed through a global perceptron based on a non-local neural network model, the foreground feature map of the training global perception charging pile will have a global image semantic association feature expression scale based on a non-local neural network model in addition to the local image semantic association feature expression scale based on the convolution kernel.

[0054] However, when the local image semantic association feature representations of different scales and the global image semantic association feature representations of the training global perception charging pile foreground feature map pass through the classifier, the training effect of the training global perception charging pile foreground feature map when passing through the classifier will be affected due to the difference in association accuracy of the local-global image semantic association features of different scales.

[0055] Therefore, in a preferred embodiment of the present application, the training global perception charging pile foreground feature map is processed using the classifier to obtain a training classification result, including:

[0056] Calculate the sum of the absolute values ​​of each eigenvalue of the training global perception charging pile foreground feature map to obtain a first global perception charging pile foreground and modulation value, and calculate the square root of the sum of the squares of each eigenvalue of the training global perception charging pile foreground feature map to obtain a second global perception charging pile foreground and modulation value;

[0057] After performing dot-wise subtraction on the training global perception charging pile foreground feature map and the second global perception charging pile foreground and modulation value, perform dot-wise multiplication on the number of eigenvalues ​​of the training global perception charging pile foreground feature map and the reciprocal of the first global perception charging pile foreground and modulation value, and take the reciprocal of each eigenvalue to obtain the first global perception charging pile foreground phase transition feature map;

[0058] After performing dot-wise subtraction on the training global perception charging pile foreground feature map and the first global perception charging pile foreground and modulation value, perform dot-wise multiplication on the square root of the number of eigenvalues ​​of the training global perception charging pile foreground feature map and the reciprocal of the second global perception charging pile foreground and modulation value, and take the reciprocal of each eigenvalue to obtain the second global perception charging pile foreground phase transition feature map;

[0059] Subtract the dot product feature map of the second global perception charging pile foreground phase transition feature map and the weighted hyperparameter from the first global perception charging pile foreground phase transition feature map to obtain an optimized training global perception charging pile foreground feature map;

[0060] The optimized trained global perception charging pile foreground feature map is passed through the classifier to obtain the classification result.

[0061] Here, the training global perception charging pile foreground feature map, denoted as F, is optimized as follows:

[0062]

[0063] f i ∈F∈R W×H×C

[0064] n=W×H×C

[0065] Wherein, F is the training global perception charging pile foreground feature map, R represents a real number set, W, H, and C represent the width, length, and height of the training global perception charging pile foreground feature map, respectively, and f i represents the feature value of the i-th position of the training global perception charging pile foreground feature map, n represents the number of feature values ​​of the training global perception charging pile foreground feature map, α represents the first global perception charging pile foreground and modulation value, β represents the second global perception charging pile foreground and modulation value, ⊙ represents the point multiplication by position, is matrix multiplication, Indicates positional subtraction, (·) ⊙-1 Represents the inverse of each eigenvalue of the calculated feature map, F1 represents the first global perception charging pile foreground phase transition feature map, F2 represents the second global perception charging pile foreground phase transition feature map, ω represents the weighted hyperparameter, and F′ represents the optimized training global perception charging pile foreground feature map.

[0066] In the preferred example, the difference of the feature value of the training global perception charging pile foreground feature map relative to the overall feature set of the training global perception charging pile foreground feature map and the modulation representation is used as the semantic change intensity information, and the phase-like conversion corresponding to the position-based intensity modulation is performed through the different modulation representation forms, so that the aggregation enhancement of the semantic change phase perception can improve the axial aggregation receptive field along the feature aggregation direction by performing the spatial translation operation based on alternating stacking under the set scale balance of the training global perception charging pile foreground feature map, thereby improving the perception effect of the aggregate semantics of the training global perception charging pile foreground feature map for the detail semantic change, so as to improve the expression effect of the training global perception charging pile foreground feature map, thereby improving the training effect of the training global perception charging pile foreground feature map when the classification regression training is performed by the classifier. In this way, it is possible to judge whether the charging pile is damaged based on the on-site photos, thereby realizing the remote intelligent monitoring and management of the charging pile, and improving the monitoring and management efficiency and service quality of the charging pile.

[0067] As described above, the device remote intelligent monitoring and management system 300 according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with a device remote intelligent monitoring and management algorithm. In a possible implementation, the device remote intelligent monitoring and management system 300 according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the device remote intelligent monitoring and management system 300 can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the device remote intelligent monitoring and management system 300 can also be one of the many hardware modules of the wireless terminal.

[0068] Alternatively, in another example, the device remote intelligent monitoring and management system 300 and the wireless terminal may also be separate devices, and the device remote intelligent monitoring and management system 300 may be connected to the wireless terminal via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0069] Furthermore, a method for remote intelligent monitoring and management of equipment is also provided.

[0070] Figure 6 FIG. 1 is a flow chart of a method for remote intelligent monitoring and management of equipment according to an embodiment of the present application. Figure 6As shown, the remote intelligent monitoring and management method for equipment according to the embodiment of the present application includes the following steps: S1, collecting a status image of a monitored charging pile through a camera; S2, transmitting the status image to a background server through a wireless transmission module; S3, on the background server, performing foreground and background segmentation on the status image to obtain a foreground object area image of the charging pile; S4, performing feature analysis on the foreground object area image of the charging pile to obtain a foreground feature of the charging pile; S5, determining whether the monitored charging pile is damaged based on the foreground feature of the charging pile.

[0071] In summary, the remote intelligent monitoring and management method of the equipment according to the embodiment of the present application is explained, which uses a camera to collect the status image of the charging pile and sends the image to the background server through a wireless transmission module. In the background server, an image processing and analysis algorithm based on machine vision is introduced to analyze the status image, so as to judge whether the charging pile is damaged based on the on-site photos, for example, parts fall off, abnormal power indicator light, etc. In this way, remote intelligent monitoring and management of the charging pile can be realized, and the monitoring and management efficiency and service quality of the charging pile can be improved.

[0072] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A remote intelligent monitoring and management system for equipment, characterized in that: include: A charging pile status image acquisition module is used to acquire status images of monitored charging piles through a camera; A state image transmission module, used to transmit the state image to a background server via a wireless transmission module; A state image semantic segmentation module, used to perform foreground and background segmentation on the state image on the backend server to obtain a charging pile foreground object area image; A charging pile foreground image feature analysis module, used for performing feature analysis on the charging pile foreground object area image to obtain charging pile foreground features; The charging pile damage detection module is used to determine whether the monitored charging pile is damaged based on the foreground characteristics of the charging pile.

2. The equipment remote intelligent monitoring and management system according to claim 1 is characterized in that: The charging pile foreground image feature analysis module includes: A charging pile foreground image feature extraction unit, used to extract features from the charging pile foreground object area image by an image feature extractor based on a deep neural network model to obtain a charging pile foreground feature map; The charging pile foreground feature enhancement unit is used to perform spatial feature enhancement and full perception encoding on the charging pile foreground feature map to obtain the charging pile foreground feature.

3. The equipment remote intelligent monitoring and management system according to claim 2 is characterized in that: The deep neural network model is a convolutional neural network model.

4. The remote intelligent monitoring and management system for equipment according to claim 3 is characterized in that: The charging pile prospect feature enhancement unit includes: A charging pile foreground feature spatial manifestation subunit, used for passing the charging pile foreground feature map through a spatial attention layer to obtain a spatially manifested charging pile foreground feature map; The charging pile feature global perception subunit is used to obtain a global perception charging pile foreground feature map as the charging pile foreground feature by passing the spatially explicit charging pile foreground feature map through a global sensor based on a non-local neural network model.

5. The equipment remote intelligent monitoring and management system according to claim 4 is characterized in that: The charging pile damage detection module is used to: pass the global perception charging pile foreground feature map through a classifier to obtain a classification result, and the classification result is used to indicate whether the monitored charging pile is damaged.

6. The equipment remote intelligent monitoring and management system according to claim 5, characterized in that: It also includes a training module for training the image feature extractor based on the convolutional neural network model, the spatial attention layer, the global perceptron based on the non-local neural network model, and the classifier.

7. The equipment remote intelligent monitoring and management system according to claim 6, characterized in that: The training module comprises: A training data acquisition unit, used to acquire training data, wherein the training data includes a training state image of a monitored charging pile and a true value of whether the monitored charging pile is damaged; A training data transmission unit, used for transmitting the training state image to the background server via a wireless transmission module; A training state image foreground and background segmentation unit, used to perform foreground and background segmentation on the training state image on the background server to obtain a training charging pile foreground object area image; A training state image foreground feature extraction unit, used for passing the training charging pile foreground object area image through the image feature extractor based on the convolutional neural network model to obtain a training charging pile foreground feature map; A training image spatial feature visualization unit, configured to pass the training charging pile foreground feature map through the spatial attention layer to obtain a training spatial visualization charging pile foreground feature map; A training image feature global perception unit, used for passing the training spatial explicit charging pile foreground feature map through the global perceptron based on the non-local neural network model to obtain a training global perception charging pile foreground feature map; A classification loss unit, used for passing the training global perception charging pile foreground feature map through the classifier to obtain a classification loss function value; A model training unit is used to train the image feature extractor based on the convolutional neural network model, the spatial attention layer, the global perceptron based on the non-local neural network model, and the classifier based on the classification loss function value and through the direction propagation of gradient descent.

8. The equipment remote intelligent monitoring and management system according to claim 7, characterized in that: The classification loss unit is used to: Using the classifier to process the training global perception charging pile foreground feature map to obtain a training classification result: and A cross entropy loss function value between the training classification result and a true value of whether the monitored charging pile is damaged is calculated as the classification loss function value.

9. A remote intelligent monitoring and management method for equipment, characterized in that: include: Collect the status image of the monitored charging pile through the camera; Transmitting the status image to a backend server via a wireless transmission module; On the backend server, segmenting the state image into foreground and background to obtain a charging pile foreground object area image; Performing feature analysis on the charging pile foreground object area image to obtain charging pile foreground features; Based on the foreground features of the charging pile, determine whether the monitored charging pile is damaged.

Images