Monitoring Method, Device, On-vehicle Terminal and Storage Medium for Special Electric Vehicles

Abstract

The present disclosure relates to the technical field of power systems, and specifically discloses a monitoring method, device, vehicle-mounted terminal and storage medium for special power vehicles. The method includes: extracting image feature vectors from the acquired image monitoring data; extracting data feature vectors from the acquired time series monitoring data; performing data analysis on the acquired text monitoring data to obtain an anomaly recognition result; using the image monitoring feature vectors as the Q matrix in the cross-attention mechanism, using the time series monitoring feature vectors as the K matrix and V matrix in the cross-attention mechanism, and calculating to obtain attention information; inputting the attention information into the monitoring model, executing the monitoring model, and obtaining partial monitoring results output by the monitoring model; splicing the partial monitoring results and the anomaly recognition result together to obtain a fused monitoring result. This technical solution can comprehensively monitor special power vehicles.

Description

Technical Field

[0001] The present disclosure relates to the technical field of power systems, and particularly to a monitoring method, device, vehicle-mounted terminal and storage medium for special power vehicles. Background Art

[0002] In recent years, the State Grid has continuously strengthened the configuration and use of special power vehicles to perform daily operations and emergency power supply tasks such as live working, emergency repair, and power supply guarantee for major events. With the increasing demand for the application of special power vehicles in live maintenance and construction operations, as well as in the operation of restoring power first and then repairing, and with the frequent occurrence of extreme weather and natural disasters, it is necessary for special power vehicles to quickly respond to dispatching commands.

[0003] The inventor has found through research that currently, special power vehicles are restricted by the imperfect digital and intelligent emergency system of the distribution network. Basically, when a problem is detected in a certain part of the vehicle, only that part is repaired. The monitoring is not comprehensive and cannot fundamentally solve the problem. Summary of the Invention

[0004] To solve the problems in the related art, embodiments of the present disclosure provide a monitoring method, device, vehicle-mounted terminal and storage medium for special power vehicles.

[0005] In a first aspect, embodiments of the present disclosure provide a monitoring method for special power vehicles, including:

[0006] Obtaining image monitoring data, time series monitoring data, and text monitoring data of the special power vehicle;

[0007] Performing image feature extraction on the image monitoring data through a preset target detection model to obtain an image monitoring feature vector;

[0008] Performing data feature extraction on the time series monitoring data through a preset anomaly detection model to obtain a time series monitoring feature vector;

[0009] Performing data analysis on the text monitoring data through a preset data anomaly recognition rule to obtain an anomaly recognition result;

[0010] Taking the image monitoring feature vector as the Q matrix in the cross-attention mechanism, taking the time series monitoring feature vector as the K matrix and V matrix in the cross-attention mechanism, and calculating to obtain attention information;

[0011] Inputting the attention information into a preset monitoring model, executing the monitoring model, and obtaining a partial monitoring result output by the monitoring model;

[0012] Concatenating the partial monitoring result and the anomaly recognition result together to obtain a fused monitoring result.

[0013] In a possible implementation manner, the method further includes:

[0014] Performing image recognition on the image monitoring feature vector through the target detection model to obtain a target image recognition result;

[0015] Performing anomaly prediction on the time series monitoring feature vector through the anomaly detection model to determine whether the data at the current moment in the time series monitoring data is abnormal.

[0016] In a possible implementation manner, the performing image recognition on the image monitoring feature vector through the target detection model to obtain a target image recognition result includes:

[0017] Performing image recognition on the image monitoring data through a first target detection model to obtain a first image recognition result;

[0018] Performing image recognition on the image monitoring data through a second target detection model to obtain a second image recognition result;

[0019] If the first image recognition result and the second image recognition result are consistent, determining the target image recognition result as the first image recognition result or the second image recognition result;

[0020] If the first image recognition result and the second image recognition result are inconsistent, continue performing image recognition until the first image recognition result and the second image recognition result are consistent, and then determine the target image recognition result as the first image recognition result or the second image recognition result; or until the number of image recognition times exceeds a predetermined number, then randomly select the first image recognition result or the second image recognition result as the target image recognition result.

[0021] In a possible implementation manner, the extracting image feature vectors from the image monitoring data through a preset target detection model to obtain image monitoring feature vectors includes:

[0022] Obtaining the image monitoring feature vectors extracted by the target detection model corresponding to the target image recognition result.

[0023] In a possible implementation manner, the obtaining image monitoring data, time series monitoring data, and text monitoring data of the electric special vehicle includes:

[0024] Obtaining the image monitoring data of the electric special vehicle through a photographing device;

[0025] Obtain the time - series monitoring data of the power special vehicle through sensors, where the time - series monitoring data includes environmental monitoring time - series data and index monitoring time - series data of predetermined important components;

[0026] Obtain the position information of the power special vehicle through a Beidou communication chip, and obtain the vehicle chassis information of the power special vehicle through a chassis monitoring device. The text monitoring data includes the position information and the vehicle chassis information.

[0027] In a possible implementation manner, the method further includes:

[0028] Input the fusion monitoring result into a pre - set abnormal monitoring decision model, execute the abnormal monitoring decision model, and obtain the decision output by the abnormal monitoring decision model;

[0029] Output the decision so that the decision can be executed.

[0030] In a second aspect, an embodiment of the present disclosure provides a monitoring device for a power special vehicle, including:

[0031] A data acquisition module, configured to acquire image monitoring data, time - series monitoring data, and text monitoring data of the power special vehicle;

[0032] A first extraction module, configured to extract image features from the image monitoring data through a pre - set target detection model to obtain an image monitoring feature vector;

[0033] A second extraction module, configured to extract data features from the time - series monitoring data through a pre - set abnormal detection model to obtain a time - series monitoring feature vector;

[0034] A data analysis module, configured to perform data analysis on the text monitoring data through a pre - set data anomaly recognition rule to obtain an anomaly recognition result;

[0035] An attention module, configured to use the image monitoring feature vector as the Q matrix in the cross - attention mechanism, use the time - series monitoring feature vector as the K matrix and V matrix in the cross - attention mechanism, and calculate to obtain attention information;

[0036] A monitoring module, configured to input the attention information into a pre - set monitoring model, execute the monitoring model, and obtain a partial monitoring result output by the monitoring model;

[0037] A fusion module, configured to splice the partial monitoring result and the anomaly recognition result together to obtain a fusion monitoring result.

[0038] In a possible implementation manner, the device further includes:

[0039] An image recognition module, configured to perform image recognition on the image monitoring feature vector through the target detection model to obtain a target image recognition result;

[0040] An anomaly prediction module, configured to perform anomaly prediction on the time series monitoring feature vector through the anomaly detection model to determine whether the data at the current moment in the time series monitoring data is abnormal.

[0041] In a possible implementation manner, the image recognition module is configured to:

[0042] Perform image recognition on the image monitoring data through a first target detection model to obtain a first image recognition result;

[0043] Perform image recognition on the image monitoring data through a second target detection model to obtain a second image recognition result;

[0044] If the first image recognition result and the second image recognition result are consistent, determine that the target image recognition result is the first image recognition result or the second image recognition result;

[0045] If the first image recognition result and the second image recognition result are inconsistent, continue to perform image recognition until the first image recognition result and the second image recognition result are consistent, and then determine that the target image recognition result is the first image recognition result or the second image recognition result; or until the number of image recognition times exceeds a predetermined number of times, randomly select the first image recognition result or the second image recognition result as the target image recognition result.

[0046] In a possible implementation manner, the first extraction module is configured to:

[0047] Obtain the image monitoring feature vector extracted by the target detection model corresponding to the target image recognition result.

[0048] In a possible implementation manner, the data acquisition module is configured to:

[0049] Obtain the image monitoring data of the special-purpose electric vehicle through a photographing device;

[0050] Obtain the time series monitoring data of the special-purpose electric vehicle through a sensor, where the time series monitoring data includes environmental monitoring time series data and index monitoring time series data of predetermined important components;

[0051] Obtain the position information of the electric special vehicle through the Beidou communication chip, and obtain the vehicle chassis information of the electric special vehicle through the chassis monitoring device. The text monitoring data includes the position information and the vehicle chassis information.

[0052] In a possible implementation manner, the device further includes:

[0053] A decision-making module, configured to input the fusion monitoring result into a pre-set abnormal monitoring decision-making model, execute the abnormal monitoring decision-making model, and obtain the decision output by the abnormal monitoring decision-making model;

[0054] An output module, configured to output the decision so that the decision can be executed.

[0055] In a third aspect, an embodiment of the present disclosure provides a vehicle-mounted terminal, including a memory and a processor. Among them, the memory is used to store one or more computer instructions, and the one or more computer instructions are executed by the processor to implement the method described in any one of the first aspects.

[0056] In a possible implementation manner, it further includes:

[0057] A power distribution security encryption unit, responsible for encrypting and decrypting power distribution service data;

[0058] A video security encryption unit, responsible for encrypting and decrypting the monitored video data;

[0059] A Beidou communication unit, responsible for communicating the position information and Beidou short message information of the electric special vehicle;

[0060] A remote communication unit, responsible for the remote communication between the vehicle-mounted terminal and the video monitoring platform and the Internet of Things management platform.

[0061] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, on which computer instructions are stored, and when the computer instructions are executed by a processor, the method described in any one of the first aspects is implemented.

[0062] According to the technical solution provided by the embodiments of the present disclosure, image monitoring data, time series monitoring data, and text monitoring data of a special-purpose electric vehicle can be obtained; image feature extraction is performed on the image monitoring data through a preset target detection model to obtain an image monitoring feature vector; data feature extraction is performed on the time series monitoring data through a preset anomaly detection model to obtain a time series monitoring feature vector; data analysis is performed on the text monitoring data through a preset data anomaly recognition rule to obtain an anomaly recognition result; the image monitoring feature vector is used as the Q matrix in the cross-attention mechanism, the time series monitoring feature vector is used as the K matrix and V matrix in the cross-attention mechanism, and attention information is calculated; the attention information is input into a monitoring model, and the monitoring model is executed to obtain a partial monitoring result output by the monitoring model; the partial monitoring result and the anomaly recognition result are spliced together to obtain a fused monitoring result; thus, by comprehensively processing the three different modalities of data, namely, image monitoring data, time series monitoring data, and text monitoring data of the special-purpose electric vehicle, a fused monitoring result is obtained, and the monitoring result is more comprehensive, so as to fundamentally solve problems based on this comprehensive monitoring result.

[0063] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In combination with the drawings, through the following detailed description of non-limiting embodiments, other features, objects, and advantages of the present disclosure will become more apparent. The following is an explanation of the drawings.

[0065] Figure 1 The flowchart showing the monitoring method of the special-purpose electric vehicle provided by the embodiments of the present disclosure is shown.

[0066] Figure 2 The schematic diagram showing the processing flow of the cross-attention mechanism provided by the embodiments of the present disclosure is shown.

[0067] Figure 3 The block diagram showing the structure of the monitoring device of the special-purpose electric vehicle provided by the embodiments of the present disclosure is shown.

[0068] Figure 4 The block diagram showing the structure of an electronic device according to an embodiment of the present disclosure is shown.

[0069] Figure 5 The schematic diagram showing the structure of a computer system suitable for implementing the method of the embodiments of the present disclosure is shown. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0070] In the following, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for clarity, parts unrelated to the description of the exemplary embodiments are omitted in the drawings.

[0071] In the present disclosure, it should be understood that terms such as "including" or "having" are intended to indicate the presence of features, numbers, steps, actions, components, parts, or combinations thereof disclosed in this specification, and are not intended to exclude the possibility of the presence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0072] It should be further noted that, without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other. The present disclosure will be described in detail below with reference to the drawings and in combination with the embodiments.

[0073] Figure 1 The flowchart showing the monitoring method of the special power vehicle provided by the embodiment of the present disclosure is as follows. As Figure 1 shown, the monitoring method of the special power vehicle includes the following steps S101 - S107:

[0074] In step S101, image monitoring data, time - series monitoring data, and text monitoring data of the special power vehicle are acquired;

[0075] In step S102, image feature extraction is performed on the image monitoring data through a preset target detection model to obtain an image monitoring feature vector;

[0076] In step S103, data feature extraction is performed on the time - series monitoring data through a preset anomaly detection model to obtain a time - series monitoring feature vector;

[0077] In step S104, data analysis is performed on the text monitoring data through a preset data anomaly recognition model to obtain an anomaly recognition result;

[0078] In step S105, the image monitoring feature vector is used as the Q matrix in the cross - attention mechanism, the time - series monitoring feature vector is used as the K matrix and V matrix in the cross - attention mechanism, and attention information is calculated;

[0079] In step S106, the attention information is input into a preset monitoring model, and the monitoring model is executed to obtain a partial monitoring result output by the monitoring model;

[0080] In step S107, the partial monitoring result and the anomaly recognition result are spliced together to obtain a fused monitoring result.

[0081] In a possible implementation, the monitoring method of the special power vehicle is applicable to an in-vehicle terminal capable of monitoring the special power vehicle.

[0082] In a possible implementation, the following three different modalities of monitoring data can be obtained through various monitoring devices mounted on the special power vehicle: image monitoring data, time series monitoring data, and text monitoring data.

[0083] Among them, the image monitoring data refers to the image data around the special power vehicle collected by a photographing device such as a camera. The time series monitoring data refers to the time series data related to the operating conditions of the special power vehicle detected by sensors. For example, the time series monitoring data can be environmental sequence data such as temperature time series data and humidity time series data near the special power vehicle, or the index sequence data of important components on the special power vehicle. For example, when the special power vehicle is a grid emergency power supply vehicle, the time series monitoring data can be index sequence data such as voltage sequence data and current sequence data of the generator. The text monitoring data refers to the data that can be described in words monitored by certain specific monitoring devices. For example, the location information of the special power vehicle, the chassis information of the special power vehicle, etc.

[0084] In a possible implementation, cross-modal fusion can be performed on the above three modalities of monitoring data to obtain a comprehensive monitoring result and achieve a comprehensive and integrated monitoring of the special power vehicle. Before fusion, corresponding data processing can be performed on these three modalities of monitoring data respectively.

[0085] In a possible implementation, for the image monitoring data, a preset object detection model can be used to extract image features from the image monitoring data to obtain an image monitoring feature vector. Here, the object detection model refers to a model that can detect objects in an image. For example, it can be an RCNN (Region-based Convolutional Neural Networks) model or a YOLO (You Only Look Once) model, etc. The input of the object detection model is image data, and the output is the objects detected in the image data. The object detection model includes a feature extraction layer and an output layer. The feature extraction layer can extract an image monitoring feature vector from the image monitoring data. After inputting the image monitoring feature vector into the output layer, the output layer can output the objects detected in the image data. Here, only the image monitoring feature vector is used, and the final output result is not used.

[0086] In a possible implementation, for time series monitoring data, a preset anomaly detection model can be used to extract data features from the time series monitoring data to obtain a time series monitoring feature vector. The anomaly detection model described here can be a model for detecting whether the monitoring data at the current moment is abnormal. For example, it can be an RNN (Recurrent Neural Network) model. The RNN model is a type of time-recursive neural network specifically used to process data with sequence or sequential dependencies. The RNN model will remember the previous information and apply it to the calculation of the current output, that is, the nodes between the hidden layers are no longer unconnected but connected, and the input of the hidden layer includes not only the output of the input layer but also the output of the hidden layer at the previous moment. The input of the anomaly detection model is the time series monitoring data, and the output is whether the monitoring data at the current moment in the time series monitoring data is abnormal. The time series monitoring data includes a feature extraction layer and an output layer. The feature extraction layer can extract a time series monitoring feature vector from the time series monitoring data. After inputting the time series monitoring feature vector into the output layer, the output layer can output whether the monitoring data at the current moment in the time series monitoring data is abnormal. Here, only the time series monitoring feature vector is used, and the final output result is not used.

[0087] In a possible implementation, for text monitoring data, data analysis can be performed on the text monitoring data through preset data anomaly recognition rules to obtain an anomaly recognition result. The preset data anomaly recognition rules can be rules preset for determining whether the text monitoring data is abnormal. When the text monitoring data is the location information of the electric special vehicle, the preset data anomaly recognition rules can be: determining whether the current location of the electric special vehicle is consistent with the location pre-specified by the system based on the location information of the electric special vehicle. If it is consistent, the anomaly recognition result can be that the electric special vehicle has not reached the specified location. If it is inconsistent, the anomaly recognition result can be that the electric special vehicle has reached the specified location. When the text monitoring data is vehicle chassis information, the preset data anomaly recognition rules can be: comparing the current vehicle chassis information with the normal vehicle chassis information. If it does not belong to the normal vehicle chassis information, the anomaly recognition result is that the vehicle condition is abnormal, and at this time, an anomaly alarm can be issued. If it belongs to the normal vehicle chassis information, the anomaly recognition result is that the vehicle condition is normal.

[0088] In a possible implementation, a cross-attention mechanism can be used to fuse the monitoring data of two modalities, namely the image monitoring feature vector and the time series monitoring feature vector, and partial monitoring results can be obtained through analysis. In the cross-attention mechanism, an input vector (such as the image monitoring feature vector) can be used as the Q (Query) matrix, and then the attention weights related to it can be calculated according to another input vector (such as the time series monitoring feature vector) as the K (Key) matrix, and these attention weights are applied to the V (Value) matrix, and finally the attention information is calculated.

[0089] Exemplarily, Figure 2 The schematic diagram of the processing flow of the cross-attention mechanism provided by the embodiments of the present disclosure is shown. As Figure 2 shown, the calculation process of this cross-attention mechanism includes:

[0090] Step 1, use the image monitoring feature vector as the Q matrix in the cross-attention mechanism, and use the time series monitoring feature vector as the K matrix and the V matrix in the cross-attention mechanism;

[0091] Step 2, calculate the similarity between the Q matrix, that is, the image monitoring feature vector, and the K matrix, that is, the time series monitoring feature vector, calculate the similarity between K and Q, and perform scaling processing on the similarity. This similarity can indicate the attention relationship between the image monitoring feature vector and the time series monitoring feature vector.

[0092] Step 3, perform softmax function normalization processing on the scaling result to obtain a probability value between 0 and 1, and this probability value represents the attention weight of the query to the key, that is, the V matrix.

[0093] Step 4, after performing a dot product operation on the and the probability value, obtain the attention information, and this attention information can be expressed by the following formula:

[0094]

[0095] Where, is the dot product operation of Q and the transpose of K ( ), indicating and similarity at different positions; is the dimension of K, used as a scaling factor to avoid too large numerical values; the softmax function converts the scaled similarity into a probability distribution; the dot product operation of V and the probability value is equivalent to extracting the concerned information from the value V to obtain the attention information .

[0096] In a possible implementation manner, the attention information is the fused data, and the attention information can be input into a monitoring model. The monitoring model makes a comprehensive determination based on the attention information to obtain a partial monitoring result output by the monitoring model. For example, the partial monitoring result can be "A cat has broken into the vicinity of the emergency power vehicle, which is somewhat dangerous. The current output current value of the generator set is: 3.3 A, the voltage value is 220.12 v, and the temperature and humidity of the surrounding environment meet the requirements."

[0097] In a possible implementation manner, the partial monitoring result obtained in step S106 and the anomaly recognition result obtained in step S103 can be concatenated to obtain a fused monitoring result. By way of example, the partial monitoring result is still as shown in the above example, and the anomaly recognition result is "The vehicle condition is good, and the vehicle has not reached the designated position." The fused monitoring result obtained by concatenating the anomaly recognition result and the partial monitoring result is "A cat has broken into the vicinity of the emergency power vehicle, which is somewhat dangerous. The current output current value of the generator set is: 3.3 A, the voltage value is 220.12 v, the vehicle condition is good, the vehicle has not reached the designated position, and the temperature and humidity of the surrounding environment meet the requirements."

[0098] This implementation manner can obtain image monitoring data, time series monitoring data, and text monitoring data of a special power vehicle; extract image feature vectors from the image monitoring data through a preset target detection model; extract data feature vectors from the time series monitoring data through a preset anomaly detection model; perform data analysis on the text monitoring data through a preset data anomaly recognition rule to obtain an anomaly recognition result; use the image monitoring feature vector as the Q matrix in the cross-attention mechanism, use the time series monitoring feature vector as the K matrix and V matrix in the cross-attention mechanism, calculate to obtain attention information; input the attention information into the monitoring model, execute the monitoring model, and obtain a partial monitoring result output by the monitoring model; concatenate the partial monitoring result and the anomaly recognition result to obtain a fused monitoring result; thus comprehensively processing the three different modalities of data, namely image monitoring data, time series monitoring data, and text monitoring data, of the special power vehicle to obtain a fused monitoring result, so that the monitoring result is more comprehensive, in order to fundamentally solve problems based on this comprehensive monitoring result.

[0099] In a possible implementation manner, the method further includes:

[0100] Performing image recognition on the image monitoring feature vector through the target detection model to obtain a target image recognition result;

[0101] Perform anomaly prediction on the time series monitoring feature vector through the anomaly detection model to determine whether the data at the current moment in the time series monitoring data is abnormal.

[0102] In this embodiment, in addition to the above-mentioned fusion monitoring results, the target detection model can also be used alone to perform image recognition on the image monitoring feature vector to obtain a target image recognition result, and the anomaly detection model is used to perform anomaly prediction on the time series monitoring feature vector to determine whether the data at the current moment in the time series monitoring data is abnormal.

[0103] In a possible embodiment, the performing image recognition on the image monitoring feature vector through the target detection model to obtain a target image recognition result includes:

[0104] Perform image recognition on the image monitoring data through a first target detection model to obtain a first image recognition result;

[0105] Perform image recognition on the image monitoring data through a second target detection model to obtain a second image recognition result;

[0106] If the first image recognition result and the second image recognition result are consistent, determine that the target image recognition result is the first image recognition result or the second image recognition result;

[0107] If the first image recognition result and the second image recognition result are inconsistent, continue with image recognition until the first image recognition result and the second image recognition result are consistent, and then determine that the target image recognition result is the first image recognition result or the second image recognition result; or until the number of image recognition times exceeds a predetermined number, randomly select the first image recognition result or the second image recognition result as the target image recognition result.

[0108] In this embodiment, the first target detection model can be an RCNN model, and the second target detection model can be a YOLO model. The YOLO model and the RCNN model are two different target detection models. The RCNN model uses a two-step method for detection, pre-generating some candidate boxes, and then classifying the content in the candidate boxes and correcting the positions of the candidate boxes. The YOLO model uses a one-step method for detection, directly predicting and classifying the candidate boxes at each position of the image without pre-generating some candidate boxes. It should be noted here that the first target detection model and the second target detection model can also be other types of target monitoring models, such as the Cascade R-CNN model, the DETR (Detection Transformer) model, and so on.

[0109] In this embodiment, the object detection can be performed on the image monitoring data based on the YOLO model and the RCNN respectively. If the image recognition results detected by the two models are the same object, the result is considered correct. For example, if the camera monitors that a cat has entered the dangerous area around the emergency power vehicle, then the cat is recognized by the RCNN model and the YOLO model respectively. If the image recognition results given are both cats, then the image recognition result is that the cat has entered the dangerous area around the emergency power vehicle. If the results recognized by the two models are inconsistent, the image recognition is performed again, and this is repeated a predetermined number of times, such as 50 times. If the results are still inconsistent, either one is selected as the image recognition result.

[0110] In a possible implementation manner, the extracting the image monitoring feature vector from the image monitoring data by using a preset object detection model includes:

[0111] Obtaining the image monitoring feature vector extracted by the object detection model corresponding to the target image recognition result.

[0112] In this embodiment, when the above two object detection models perform image recognition on the image monitoring data respectively, they will first extract the corresponding image monitoring feature vectors from the image monitoring data, and then perform image recognition according to the extracted image monitoring feature vectors. If the first image recognition result recognized by a certain object detection model, such as the first object detection model, is finally obtained as the image recognition result, then the first image monitoring feature vector extracted by the first object detection model can be obtained as the image monitoring feature vector.

[0113] In a possible implementation manner, the obtaining the image monitoring data, time series monitoring data, and text monitoring data of the electric special vehicle includes:

[0114] Obtaining the image monitoring data of the electric special vehicle through a photographing device;

[0115] Obtaining the time series monitoring data of the electric special vehicle through a sensor, where the time series monitoring data includes environmental monitoring time series data and index monitoring time series data of predetermined important components;

[0116] Obtaining the position information of the electric special vehicle through a Beidou communication chip, and obtaining the vehicle chassis information of the electric special vehicle through a chassis monitoring device, where the text monitoring data includes the position information and the vehicle chassis information.

[0117] In this embodiment, the photographing device can be an image sensor such as a camera. The photographing device can be installed on the special-purpose electric vehicle to photograph the images around the special-purpose electric vehicle and send the photographed images to the vehicle-mounted terminal through a PoE (Power over Ethernet) interface. In this way, the vehicle-mounted terminal can obtain the image monitoring data of the special-purpose electric vehicle.

[0118] In this embodiment, the environmental monitoring time series data can be the time series data of at least one environmental parameter such as the temperature time series, the humidity time series, and the smoke concentration time series. Among them, the temperature time series can be collected by a temperature sensor, the humidity time series can be collected by a humidity sensor, and the smoke concentration time series can be collected by a smoke sensor. These sensors can be communicatively connected to the vehicle-mounted terminal through an RS485-1 interface (a standard serial communication interface). In this way, the vehicle-mounted terminal can obtain the environmental monitoring time series data through the RS485-1 interface.

[0119] In this embodiment, different types of special-purpose electric vehicles may have different predetermined important components. For example, when the special-purpose electric vehicle is a grid emergency power vehicle, the predetermined important component may be a generator set; when the special-purpose electric vehicle is a mobile energy storage vehicle, the predetermined important component may be an energy storage component, and so on. For the predetermined important component, the corresponding indicators of the predetermined important component can be monitored to obtain the index monitoring time series data of the predetermined important component. For example, if the predetermined important component is a generator set, its corresponding indicators may be generator set index information such as voltage, current, power, frequency, power generation, circuit breaker opening and closing position, oil pressure, coolant temperature, working duration, battery voltage, etc. By monitoring these indicators, the corresponding index monitoring time series data can be obtained. The generator set can be connected to the generator set and its controller through an RS485-3 interface. In this way, the vehicle-mounted terminal can obtain the index monitoring time series data of the generator set through the RS485-3 interface.

[0120] In this embodiment, the Beidou communication chip can be set in the vehicle-mounted terminal. The vehicle-mounted terminal can obtain the position information of the special-purpose electric vehicle through the Beidou communication chip. The position information includes longitude, latitude, and altitude. In addition to communicating the position information, the Beidou communication chip is also responsible for communicating Beidou short message information, etc. The Beidou short message information includes the uplink of key information in the case of 4G / 5G anomalies. In this way, for the mobile operation scenario, on the basis of the general 4G / 5G, the functions of Beidou positioning and Beidou short message are added to improve the full-scenario communication ability of the terminal, improve the unified scheduling and command ability, and improve the safety and reliability of the vehicle.

[0121] In this killing method, the chassis monitoring device can be set in the vehicle chassis, and the vehicle chassis information can be monitored through the chassis monitoring device. The vehicle chassis information includes: engine speed, oil level, vehicle speed, fault codes and other information. The vehicle chassis can be communicatively connected to the in-vehicle terminal through a CAN (Controller Area Network) bus. In this way, the in-vehicle terminal can obtain the vehicle chassis information monitored by the chassis monitoring device through the CAN bus.

[0122] In a possible implementation manner, the method further includes:

[0123] Input the fusion monitoring result into a pre-set abnormal monitoring decision model, execute the abnormal monitoring decision model, and obtain the decision output by the abnormal monitoring decision model;

[0124] Output the decision so that the decision can be executed.

[0125] In this implementation manner, the abnormal monitoring decision model can be pre-trained by other devices and then set in the in-vehicle terminal, which is used to analyze the fusion monitoring result to obtain the corresponding decision for the fusion monitoring result. The sample data for training can be the fusion monitoring results and historical correct decisions within a historical time period. The in-vehicle terminal can use the fusion monitoring result as the input of the abnormal monitoring decision model. Finally, the abnormal monitoring decision model generates and outputs the decision. In this way, the in-vehicle terminal, as the core node, can perform local processing and make local decisions in a timely manner in case of emergencies, meeting the high-efficiency command requirements of the emergency supply support task.

[0126] In this implementation manner, the in-vehicle terminal can output the decision in ways such as voice, text, and pictures to notify the local staff to execute the decision in a timely manner.

[0127] The present disclosure also provides a monitoring device for a special power vehicle, Figure 3 showing a structural block diagram of the monitoring device for a special power vehicle provided by an embodiment of the present disclosure. The device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. As Figure 3 shown, the monitoring device for the special power vehicle includes:

[0128] A data acquisition module 301, configured to acquire image monitoring data, time series monitoring data, and text monitoring data of the special power vehicle;

[0129] A first extraction module 302, configured to perform image feature extraction on the image monitoring data through a pre-set target detection model to obtain an image monitoring feature vector;

[0130] A second extraction module 303, configured to extract data features from the time series monitoring data through a preset anomaly detection model to obtain a time series monitoring feature vector;

[0131] A data analysis module 304, configured to perform data analysis on the text monitoring data through a preset data anomaly recognition rule to obtain an anomaly recognition result;

[0132] An attention module 305, configured to use the image monitoring feature vector as the Q matrix in the cross-attention mechanism, use the time series monitoring feature vector as the K matrix and V matrix in the cross-attention mechanism, and calculate to obtain attention information;

[0133] A monitoring module 306, configured to input the attention information into a monitoring model, execute the monitoring model, and obtain a partial monitoring result output by the monitoring model;

[0134] A fusion module 307, configured to splice the partial monitoring result and the anomaly recognition result together to obtain a fusion monitoring result.

[0135] In a possible implementation manner, the device further includes:

[0136] An image recognition module, configured to perform image recognition on the image monitoring feature vector through the target detection model to obtain a target image recognition result;

[0137] An anomaly prediction module, configured to perform anomaly prediction on the time series monitoring feature vector through the anomaly detection model to determine whether the data at the current moment in the time series monitoring data is abnormal.

[0138] In a possible implementation manner, the image recognition module is configured to:

[0139] Perform image recognition on the image monitoring data through a first target detection model to obtain a first image recognition result;

[0140] Perform image recognition on the image monitoring data through a second target detection model to obtain a second image recognition result;

[0141] If the first image recognition result and the second image recognition result are consistent, determine that the target image recognition result is the first image recognition result or the second image recognition result;

[0142] If the first image recognition result and the second image recognition result are inconsistent, continue with image recognition until the first image recognition result and the second image recognition result are consistent, and then determine the target image recognition result as the first image recognition result or the second image recognition result; or until the number of image recognition times exceeds a predetermined number, randomly select the first image recognition result or the second image recognition result as the target image recognition result.

[0143] In a possible implementation manner, the first extraction module is configured to:

[0144] Obtain the image monitoring feature vector extracted by the target detection model corresponding to the target image recognition result.

[0145] In a possible implementation manner, the data acquisition module is configured to:

[0146] Obtain the image monitoring data of the electric special vehicle through an imaging device;

[0147] Obtain the time series monitoring data of the electric special vehicle through a sensor, where the time series monitoring data includes environmental monitoring time series data and index monitoring time series data of predetermined important components;

[0148] Obtain the position information of the electric special vehicle through a Beidou communication chip, and obtain the vehicle chassis information of the electric special vehicle through a chassis monitoring device, where the text monitoring data includes the position information and the vehicle chassis information.

[0149] In a possible implementation manner, the device further includes:

[0150] A decision-making module, configured to input the fusion monitoring result into a pre-set anomaly monitoring decision-making model, execute the anomaly monitoring decision-making model, and obtain the decision output by the anomaly monitoring decision-making model;

[0151] An output module, configured to output the decision so that the decision can be executed.

[0152] The technical terms and technical features mentioned in the embodiments of this device are the same as or similar to those mentioned in the above method embodiments. For the explanations and descriptions of the technical terms and technical features involved in this device, reference can be made to the explanations and descriptions of the above method embodiments, which will not be elaborated here.

[0153] This disclosure also discloses an electronic device, Figure 4 showing a structural block diagram of an electronic device according to an embodiment of this disclosure.

[0154] As Figure 4As shown, the electronic device 400 includes a memory 401 and a processor 402. Among them, the memory 401 is used to store one or more computer instructions, and the one or more computer instructions are executed by the processor 402 to implement the method according to the embodiments of the present disclosure.

[0155] In this embodiment, the vehicle-mounted terminal can obtain data such as image monitoring data, time series monitoring data, and text monitoring data provided by various monitoring devices on the special-purpose electric vehicle. As the core node, the vehicle-mounted terminal can perform fusion processing on these data to obtain a fusion monitoring result. If an emergency occurs and there is no time to communicate with the platform, the vehicle-mounted terminal can also directly make local decisions, giving full play to the core function of the edge computing node.

[0156] In a possible embodiment, the vehicle-mounted terminal further includes:

[0157] A power distribution security encryption unit, which is responsible for encrypting and decrypting power distribution service data;

[0158] A video security encryption unit, which is responsible for encrypting and decrypting the monitored video data;

[0159] A Beidou communication unit, which is responsible for the communication of the position information and Beidou short message information of the special-purpose electric vehicle;

[0160] A remote communication unit, which is responsible for the remote communication between the vehicle-mounted terminal and the video monitoring platform and the Internet of Things management platform.

[0161] In this embodiment, the vehicle-mounted terminal can obtain the monitored video data through a photographing device. After encrypting the video data using the video security encryption unit, it is sent to the video monitoring platform through the remote communication unit to ensure the security of the video data. The video monitoring platform can use the video data returned by each vehicle-mounted terminal to perform abnormal video monitoring, and send an alarm message to the corresponding vehicle-mounted device in time when an abnormal video is detected.

[0162] In this embodiment, the vehicle-mounted terminal can also transmit and interact with the Internet of Things management platform for power distribution service data through the remote communication unit. When transmitting the power distribution service data, the power distribution security encryption unit can be used for encryption and decryption to ensure the data security of the power distribution service data.

[0163] In this embodiment, the Beidou communication unit can communicate with satellites to obtain the location information of the special power vehicle. It can also be responsible for the communication of Beidou short message information in the event of abnormal 4G / 5G. The Beidou short message information includes the uplink of key information in the event of abnormal 4G / 5G. In this way, based on the general 4G / 5G, Beidou positioning and Beidou short message are added to improve the full-scenario communication ability of the terminal, enhance the unified dispatching and command ability, and improve the safety and reliability of the emergency power vehicle.

[0164] In this embodiment, by installing vehicle-mounted terminals, cameras, environmental monitoring sensors and other monitoring devices on the special power vehicle to collect relevant monitoring data of the special power vehicle, it is possible to sense and upload key operation data such as vehicle location, operation and maintenance personnel, and working status. Through the remote communication unit of the vehicle-mounted terminal, it can interact with the command system in real time, meeting the requirements of real-time monitoring of information such as the status of special power vehicles and the progress of emergency repair and power restoration online, comprehensively improving the emergency support perception and auxiliary decision-making ability of pre-event warning, in-event monitoring, and post-event review, and meeting the high-efficiency command requirements of emergency power supply support tasks.

[0165] Figure 5 The structural diagram of a computer system suitable for implementing the method of the embodiment of the present disclosure is shown.

[0166] As Figure 5 shown, the computer system 500 includes a processing unit 501, which can execute various processes in the above embodiments according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage section 508 into the random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the computer system 500 are also stored. The processing unit 501, ROM 502, and RAM 503 are connected to each other via a bus 504. The input / output (I / O) interface 505 is also connected to the bus 504.

[0167] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. The drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that the computer program read from it can be installed into the storage section 508 as needed. Among them, the processing unit 501 can be implemented as a processing unit such as a CPU, GPU, TPU, FPGA, NPU, etc.

[0168] In particular, according to embodiments of the present disclosure, the methods described above can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product that includes computer instructions which, when executed by a processor, implement the method steps described above. In such an embodiment, the computer program product can be downloaded and installed from a network via the communication section 509, and / or installed from the removable medium 511.

[0169] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functions and operations of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions denoted in the blocks may occur in an order different from that denoted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or by a combination of dedicated hardware and computer instructions.

[0170] The units or modules involved in the embodiments described in the present disclosure can be implemented in software, or can be implemented by programmable hardware. The described units or modules can also be provided in a processor, and the names of these units or modules do not constitute a limitation on the units or modules themselves in some cases.

[0171] As another aspect, the present disclosure also provides a computer-readable storage medium, which can be the computer-readable storage medium included in the electronic device or computer system in the above embodiments; or can exist separately and not be assembled into the device. The computer-readable storage medium stores one or more programs, and the one or more programs are used by one or more processors to execute the methods described in the present disclosure.

[0172] The above description is only a preferred embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) disclosed in the present disclosure that have similar functions.

Claims

1. A monitoring method for special electric vehicles, characterized in that, Including: Obtaining image monitoring data, time series monitoring data, and text monitoring data of a special power vehicle; Performing image feature extraction on the image monitoring data through a preset target detection model to obtain an image monitoring feature vector; Performing data feature extraction on the time series monitoring data through a preset anomaly detection model to obtain a time series monitoring feature vector; Performing data analysis on the text monitoring data through a preset data anomaly recognition rule to obtain an anomaly recognition result; Using the image monitoring feature vector as the Q matrix in the cross-attention mechanism, using the time series monitoring feature vector as the K matrix and V matrix in the cross-attention mechanism, and calculating to obtain attention information; Inputting the attention information into a preset monitoring model, executing the monitoring model, and obtaining a partial monitoring result output by the monitoring model; Concatenating the partial monitoring result and the anomaly recognition result to obtain a fused monitoring result.

2. The method according to claim 1, characterized in that, The method further includes: Performing image recognition on the image monitoring feature vector through the target detection model to obtain a target image recognition result; Performing anomaly prediction on the time series monitoring feature vector through the anomaly detection model to determine whether the data at the current moment in the time series monitoring data is abnormal.

3. The method according to claim 2, wherein The performing image recognition on the image monitoring feature vector through the target detection model to obtain a target image recognition result includes: Performing image recognition on the image monitoring data through a first target detection model to obtain a first image recognition result; Performing image recognition on the image monitoring data through a second target detection model to obtain a second image recognition result; If the first image recognition result and the second image recognition result are consistent, determining the target image recognition result as the first image recognition result or the second image recognition result; If the first image recognition result and the second image recognition result are inconsistent, continuing image recognition until the first image recognition result and the second image recognition result are consistent, and then determining the target image recognition result as the first image recognition result or the second image recognition result; or until the number of image recognition times exceeds a predetermined number, randomly selecting the first image recognition result or the second image recognition result as the target image recognition result.

4. The method according to claim 3, wherein The performing image feature extraction on the image monitoring data through a preset target detection model to obtain an image monitoring feature vector includes: Obtaining an image monitoring feature vector extracted by the target detection model corresponding to the target image recognition result.

5. The method according to claim 1, characterized in that The obtaining image monitoring data, time series monitoring data, and text monitoring data of a special power vehicle includes: Obtaining the image monitoring data of the special power vehicle through a photographing device; Obtaining the time series monitoring data of the special power vehicle through a sensor, where the time series monitoring data includes environmental monitoring time series data and index monitoring time series data of predetermined important components; Obtain the position information of the electric special vehicle through the Beidou communication chip, and obtain the vehicle chassis information of the electric special vehicle through the chassis monitoring device. The text monitoring data includes the position information and the vehicle chassis information.

6. The method according to claim 1, wherein The method further includes: Input the fusion monitoring result into a pre-set anomaly monitoring decision model, execute the anomaly monitoring decision model, and obtain the decision output by the anomaly monitoring decision model; Output the decision so that the decision can be executed.

7. A monitoring device for a special electric vehicle, characterized in that, Includes: A data acquisition module configured to acquire image monitoring data, time series monitoring data, and text monitoring data of an electric special vehicle; A first extraction module configured to perform image feature extraction on the image monitoring data through a pre-set target detection model to obtain an image monitoring feature vector; A second extraction module configured to perform data feature extraction on the time series monitoring data through a pre-set anomaly detection model to obtain a time series monitoring feature vector; A data analysis module configured to perform data analysis on the text monitoring data through a pre-set data anomaly recognition rule to obtain an anomaly recognition result; An attention module configured to use the image monitoring feature vector as the Q matrix in the cross-attention mechanism, use the time series monitoring feature vector as the K matrix and V matrix in the cross-attention mechanism, and calculate to obtain attention information; A monitoring module configured to input the attention information into a pre-set monitoring model, execute the monitoring model, and obtain a partial monitoring result output by the monitoring model; A fusion module configured to splice the partial monitoring result and the anomaly recognition result together to obtain a fusion monitoring result.

8. The device according to claim 7, characterized in that The device further includes: An image recognition module configured to perform image recognition on the image monitoring feature vector through the target detection model to obtain a target image recognition result; An anomaly prediction module configured to perform anomaly prediction on the time series monitoring feature vector through the anomaly detection model to determine whether the data at the current moment in the time series monitoring data is abnormal.

9. The device according to claim 8, characterized in that, The image recognition module is configured to: Perform image recognition on the image monitoring data through a first target detection model to obtain a first image recognition result; Perform image recognition on the image monitoring data through a second target detection model to obtain a second image recognition result; If the first image recognition result and the second image recognition result are consistent, determine that the target image recognition result is the first image recognition result or the second image recognition result; If the first image recognition result and the second image recognition result are inconsistent, continue with image recognition until the first image recognition result and the second image recognition result are consistent, and then determine that the target image recognition result is the first image recognition result or the second image recognition result; or until the number of image recognition times exceeds a predetermined number, randomly select the first image recognition result or the second image recognition result as the target image recognition result.

10. The device according to claim 9, characterized in that, The first extraction module is configured to: Obtain the image monitoring feature vector extracted by the target detection model corresponding to the target image recognition result.

11. The device according to claim 7, characterized in that, The data acquisition module is configured to: Obtain the image monitoring data of the special power vehicle through the photographing device; Obtain the time series monitoring data of the special power vehicle through the sensor, and the time series monitoring data includes environmental monitoring time series data and index monitoring time series data of predetermined important components; Obtain the position information of the special power vehicle through the Beidou communication chip, and obtain the vehicle chassis information of the special power vehicle through the chassis monitoring device, and the text monitoring data includes the position information and the vehicle chassis information.

12. The device according to claim 7, wherein The device further includes: A decision module, configured to input the fusion monitoring result into a pre-set abnormal monitoring decision model, execute the abnormal monitoring decision model, and obtain the decision output by the abnormal monitoring decision model; An output module, configured to output the decision so that the decision can be executed.

13. A vehicle-mounted terminal, characterized in that, It includes a memory and a processor, and the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method according to any one of claims 1 to 6.

14. The vehicle-mounted terminal according to claim 13, wherein It further includes: A power distribution security encryption unit, responsible for encrypting and decrypting power distribution service data; A video security encryption unit, responsible for encrypting and decrypting the monitored video data; A Beidou communication unit, responsible for the communication of the position information and Beidou short message information of the special power vehicle; A remote communication unit, responsible for the remote communication between the vehicle-mounted terminal and the video monitoring platform and the Internet of Things management platform.

15. A readable storage medium, characterized in that, It stores computer instructions thereon, and when the computer instructions are executed by the processor, the method according to any one of claims 1 to 6 is implemented.

Images